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HUMAN  FACTORS  RESEARCH  IN  AIRCREW  PERFORMANCE  AND  TRAINING: 

ANNUAL  SUMMARY  REPORT 


EXECUTIVE  SUMMARY 


Requirement : 

Anacapa  Sciences,  Inc.,  has  provided  collocated  research 
support  to  the  U.S.  Army  Research  Institute  Aviation  Research 
and  Development  Activity  (ARIARDA)  at  Fort  Rucker,  Alabama, 
since  1981.  The  ARIARDA  program  supports  the  full  range  of 
Army  aviation  research  requirements  with  projects  that 
address  emerging  aviation  weapon  systems,  aviation  manpower 
and  personnel  programs,  and  aviator  training  programs.  This 
Annual  Summary  Report  fulfills  one  of  the  requirements  of 
Contract  MDA903-87-C-0523 .  It  describes  the  17  research 
projects  in  the  ARIARDA  program  conducted  by  Anacapa 
Sciences,  Inc.,  researchers  between  February  1986  and  October 
1987.  The  specific  requirements  that  led  to  the  initiation 
of  each  research  project  is  discussed  in  the  individual 
summaries . 


Methods : 

There  are  substantial  differences  in  the  methods  that 
were  employed  in  the  individual  projects  and  in  each  of  the 
three  research  areas.  For  some  projects,  the  research 
approach  was  a  scientific  experiment  in  which  selected  vari¬ 
ables  were  controlled,  manipulated,  and  measured.  For  other 
projects,  the  research  approach  was  a  set  of  analytical  or 
product  development  tasks.  The  research  methods  used  in  each 
project  are  described  in  moderate  detail  in  the  individual 
summaries . 


Summary  of  Contents : 

The  research  projects  were  conducted  in  all  three  areas 
of  the  ARIARDA  program.  Four  of  the  projects  were  in  the 
emerging  systems  area:  LHX  workload  prediction,  AH-64 
workload  prediction,  MH-60K  and  MH-47E  workload  prediction, 
and  the  design  of  flight  symbology.  Five  of  the  projects 
were  in  the  manpower  and  personnel  area:  development  of  a 
new  aviator  selection  test,  evaluation  of  an  enlisted  aviator 
program,  development  of  a  peer  assessment  method,  evaluation 


of  the  First  Army  Reserve  aviation  management  method,  and  an 
evaluation  of  management  policies  for  AH-64  aviators.  The 
remaining  eight  projects  were  in  the  training  area:  eval¬ 
uating  the  training  requirements  for  National  Guard  and  Army 
Reserve  aviators,  studying  aviator  skill  decay  and  reacquisi¬ 
tion,  evaluating  aviation  gunnery  training,  evaluating  flight 
simulator  training  in  operational  Army  aviation  units, 
upgrading  the  basic  map  interpretation  and  terrain  analysis 
course  to  videodisc,  developing  an  AH-64  symbology  training 
program,  and  evaluating  the  effectiveness  of  aviation  part- 
task  trainers. 


Utilization : 

The  results  and  recommendations  of  many  of  the  17 
projects  will  be  directly  implemented  in  the  design  of  new 
aviation  systems,  in  the  selection  and  management  of  aviation 
personnel,  and  in  aviation  training  a:  the  Aviation  Center  at 
Fort  Rucker  and  in  world-wide  Army  aviation  units.  This 
report  provides  Army  weapon  system  managers,  manpower  and 
personnel  planners,  training  system  developers  and  managers, 
and  other  researchers  working  in  related  fields  with  a 
readable  summary  of  the  research  activities  in  their 
respective  areas  of  interest  . 
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HUMAN  FACTORS  RESEARCH  IN  AIRCREW  PERFORMANCE  AND  TRAINING: 

ANNUAL  SUMMARY  REPORT 


INTRODUCTION 

Anacapa  Sciences,  Inc.,  has  provided  collocated  research 
support  to  the  U.S.  Army  Research  Institute  Aviation  Research 
and  Development  Activity  ( ARIARDA)  at  Fort  Rucker,  Alabama, 
under  a  series  of  contracts  that  began  1  September  1981.  One 
of  the  requirements  of  the  current  contract  (No.  MDA903-87-C- 
0523),  which  began  on  9  October  1986,  is  to  submit  an  Annual 
Summary  Report  of  research  project  activities.  To  fulfill 
that  requirement,  this  report  describes  the  research  projects 
on  which  Anacapa  personnel  have  worked  during  the  first  year 
of  the  current  contract .  This  report  also  describes  research 
that  was  conducted  under  the  immediately  preceding  contract 
(No.  MDA903-86-C-0055)  in  effect  between  18  February  and  8 
October  1986.  The  29  research  projects  performed  by  Anacapa 
under  the  original  contract  (No.  MDA903-81-C-0504 ) ,  that 
terminated  31  December  1985,  were  described  in  a  Final 
Summary  Report  (Cross  &  Szabo,  1986)  that  was  submitted  to 
ARIARDA  in  February  of  1986. 

This  report  contains  summary  descriptions  of  the  17 
research  projects  that  Anacapa  personnel  have  worked  on  under 
both  Contracts  MDA903-81-C-0504  and  MDA903-86-C-0055  .  Most 
of  the  project  summaries  follow  the  same  general  format. 

Each  summary  begins  with  a  background  section  that  presents 
information  needed  to  understand  the  requirement  for  the 
project.  The  background  may  include  a  brief  review  of 
relevant  research  or  describe  the  critical  events  that  led  to 
the  project's  initiation.  When  the  need  for  the  research 
cannot  be  clearly  inferred  from  the  background,  a  statement 
of  need  or  definition  of  the  problem  is  presented.  The 
background  section  is  followed  by  a  concise  statement  of  the 
project  objectives. 

Next,  the  research  approach  section  presents  a  moder¬ 
ately  detailed  description  of  the  activities  that  were 
performed  to  accomplish  the  project  objectives.  For  some 
projects,  the  research  approach  is  a  scientific  experiment  in 
which  selected  variables  are  controlled,  manipulated,  and 
measured.  For  other  projects,  the  research  approach  is  a  set 
of  analytical  or  product  development  tasks.  The  research 
approach  is  followed  by  one  or  more  sections  that  present  the 
research  findings  or,  in  the  case  of  product  development 
efforts,  a  summary  description  of  the  research  products. 
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The  final  section  of  each  project  summary,  entitled 
"Project  Status, "  describes  the  work  accomplished  and  the 
work  projected,  if  any.  Where  possible,  ,  his  section  also 
presents  the  current  estimates  of  project  milestones. 

Anacapa  personnel  also  provided  temporary  support  on 
other  p.-ojects  that  are  the  primary  responsibility  of  ARIARDA 
personnel  and  are,  consequently,  not  summarized  in  this 
report.  Most  notable  among  the  support  tasks  is  work  being 
conducted  to  develop  a  training  research  simulator.  It  is 
also  important  to  note  that  the  projects  summarized  in  this 
report  represent  only  a  portion  of  the  research  program  at 
ARIARDA.  Numerous  projects  are  being  conducted  as  the  sole 
responsibility  of  ARIARDA  personnel  or  under  other  contracts. 

The  project  summaries  are  presented  in  three  content 
categories  that  reflect  the  research  emphases  at  ARIARDA. 

The  categories  are  not  rigidly  defined  because  the  content 
areas  may  overlap  and  the  projects  may  belong  to  more  than 
one  category.  Nonetheless,  this  organization  is  intended  to 
assist  the  reader  in  locating  a  specific  project  summary 
within  a  research  area  or  to  find  summaries  that  are  closely 
related  in  terms  of  content. 

The  first  four  summaries  describe  research  in  the  area 
of  emerging  systems.  The  next  five  summaries  present  work  in 
the  area  of  manpower  and  personnel  research.  The  next  eight 
summaries  are  in  the  training  research  area.  The  number  of 
summaries  assigned  to  the  various  categories  is  not  neces¬ 
sarily  in  proportion  to  the  emphasis  that  is  placed  by 
ARIARDA  on  each  content  area. 

Although  each  summary  identifies  the  project  di  ec- 
tor(s),  the  Anacapa  approach  to  our  research  efforts  at  Fort 
Rucker  is  a  team  concept.  This  approach  permits  the  maximum 
utilization  of  the  scientific  staff  members'  skills  and 
ensures  coordination  among  closely  related  projects.  The 
scientific  staff  are  ably  supported  by  an  exceptionally 
efficient  administrative  and  technical  staff.  All  the  work 
is,  of  course,  closely  coordinated  with  ARIARDA  personnel. 


VALIDATION  OF  THE  LHX  WORKLOAD  PREDICTION  MODEL 
Mr.  Theodore  B.  Aldrich,  Project  Director 


Background 

The  Army's  Air/Land  Battle  2000  scenario  represents  a 
high-threat  environment  that  will  place  heavy  workload 
demands  on  Army  aviators.  Accordingly,  future  aircraft 
systems  are  being  designed  with  advanced  technology  to 
automate  many  of  the  functions  traditionally  performed  by 
crewmembers.  Examples  of  the  advanced  technology  include: 

•  an  increased  number  of  sensors  and  target  acquisition 
aids , 

•  improved  navigation  and  communication  systems, 

•  advanced  crew  station  design  features, 

•  improved  flight  controls, 

•  exceptionally  high  avionics  reliability,  and 

•  subsystems  that  are  automatically  reconfigured  if 
components  fail. 

Although  advanced  technology  is  typically  designed  to 
reduce  aviator  workload,  the  tasks  required  to  use  the  tech¬ 
nology  may  actually  increase  workload  in  some  instances.  For 
example,  technology  designed  to  reduce  an  aviator's  need  to 
maintain  physical  control  of  system  functions  often  increases 
the  aviator's  role  as  a  systems  monitor  or  problem  solver. 
Consequently,  while  psychomotor  workload  demands  are 
decreased,  sensory  and  cognitive  attentional  demands  may  be 
increased . 

The  development  of  new  and  improved  aircraft  systems 
also  presents  problems  in  the  prediction  and  assessment  of 
operator  workload.  Metrics  that  are  appropriate  for 
analyzing  physical  workload  are  inadequate  for  assessing 
sensory  and  cognitive  workload.  Accordingly,  workload 
research  has  shifted  from  a  focus  on  physical  effort  required 
to  perform  a  task  to  an  emphasis  on  the  attentional  demand 
associated  with  the  sensory,  cognitive,  and  psychomotor 
workload  components  of  the  tasks. 

In  response  to  the  shifting  emphasis,  Anacapa  Sciences, 
Inc.,  researchers,  under  a  previous  contract  to  the  U.S.  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
(ARIARDA) ,  developed  a  methodology  for  predicting  aviator 
workload  in  advance  of  aircraft  system  design.  The  method¬ 
ology  features  models  that  predict  workload  under  varying 
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automation  configurations  and  for  both  single-  and  multi- 
crewmember  system  designs.  The  workload  prediction  method¬ 
ology  operationally  defines  workload  in  terms  of  attentional 
demand  and  predicts  workload  associated  with  task 
performance . 

To  apply  the  methodology  to  aircraft  systems,  three 
major  phases  of  research  must  be  performed.  The  three  phases 
are : 

•  conduct  mission  and  task  analyses  of  critical  mission 
segments  and  assign  estimates  of  workload  for  the 
sensory,  cognitive,  and  psychomotor  workload 
components  of  each  task; 

•  develop  computer-based  workload  prediction  models 
using  the  data  produced  by  the  task  analyses;  and 

•  exercise  the  computer  models  to  produce  predictions  of 
crew  workload  under  varying  automation  and  crew 
configurations . 

Under  the  previous  ARIARDA  contract,  Anacapa  researchers 
performed  the  task  analyses,  developed  both  one-  and  two- 
crewmember  workload  prediction  models,  and  applied  them  to  a 
proposed  multipurpose,  lightweight  helicopter  designated  the 
LHX .  A  summary  description  of  the  work  accomplished  in  each 
of  the  three  phases  is  presented  below.  A  more  detailed 
description  of  the  methodology  and  its  applications  to  the 
LHX  is  presented  in  Aldrich,  Szabo,  and  Craddock  (1985) 


Phase  1;  Conduct  a  Mission  and  Task  Analysis 

During  Phase  1,  the  Anacapa  researchers  conducted  a 
mission  and  task  analysis  for  the  proposed  LHX.  The  analysis 
used  a  top-down  approach  in  which  LHX  mission  profiles  were 
subdivided  into  mission  phases  and,  subsequently,  into 
mission  segments.  A  segment  is  defined  as  a  major  sequence 
of  events  that  has  a  definite  start  and  end  point.  The 
events  in  a  segment  may  occur  concurrently  or  sequentially. 

Each  segment  was  then  divided  into  functions.  A  function 
is  defined  as  a  set  of  activities  that  must  be  performed  eithe 
by  an  operator  or  by  equipment  to  complete  a  portion  of  the 
mission  segment.  Functions  were  categorized  as  flight  control 
support,  or  mission,  and  placed  on  segment  timelines. 

Finally,  the  functions  for  each  segment  were  divided 
into  tasks.  Each  task  is  a  specific  crew  activity  that  is 
essential  to  the  successful  performance  of  the  function.  The 
task  consists  of  a  verb  and  an  object  and  was  analyzed  to: 


•  identify  the  crewmember (s)  performing  the  task, 

•  identify  the  subsystem  representing  the  primary  man- 
machine  interface, 

•  estimate  the  workload  imposed  on  the  crewmember (s) , 
and 

•  estimate  the  time  required  to  complete  the  task. 

The  crewmember ( s )  performing  each  task  and  the  sub¬ 
systems  associated  with  each  task  were  identified  by 
examining  the  performance  of  similar  tasks  in  existing  Army 
helicopters.  Short  verbal  descriptors  of  the  attentional 
demand  requirements  for  each  task  were  written  for  each 
workload  component.  The  descriptors  were  then  compared  with 
the  verbal  anchors  on  the  visual,  auditory,  cognitive,  and 
psychomotor  workload  component  rating  scales.  The  ratings 
associated  with  the  anchors  that  best  matched  the  verbal 
descriptors  were  assigned  as  the  numerical  estimates  of  work¬ 
load.  Two  or  more  analysts  performed  the  ratings  indepen¬ 
dently  and  then  reached  a  consensus  on  the  final  ratings  for 
each  task.  Task  time  estimates  were  assigned  on  the  basis  of 
interviews  with  subject  matter  experts  (SMEs) . 


Phase  2  ; _ Develop  Computer-Based  Workload  Prediction  Models 

A  bottom-up  approach,  in  which  the  tasks  identified  in 
Phase  1  served  as  the  basic  elements  of  analysis,  was  used  to 
develop  the  computer  models  for  predicting  workload.  First, 
the  analytic  information  for  each  task  was  entered  into 
computer  data  files.  Then,  time-based  decision  rules  were 
programmed  to  build  functions  from  the  tasks  and,  subse¬ 
quently,  to  build  segments  from  the  functions.  The  decision 
rules  define  the  temporal  relationships  among  tasks  and 
functions  as  determined  in  the  Phase  1  mission  and  task 
analysis . 

The  computer  models  produce  estimates  of  total  workload 
at  half-second  intervals  for  each  workload  component  (i.e. 
visual,  auditory,  cognitive  and  psychomotor) .  The  estimates 
are  derived  by  summing  the  ratings  assigned  to  each  workload 
component  across  concurrent  tasks.  A  total  value  of  "8"  on 
any  single  half-second  timeline  constitutes  the  threshold  for 
an  overload  within  a  given  workload  component. 


Phase  3:  Exercise  the  Comouter  Models 


During  Phase  3,  the  computer  models  were  exercised  to 
predict  workload  associated  with  individual  automation 


options  and  combinations  of  options.  Three  steps  were 
performed  to  produce  the  workload  predictions: 

•  select  the  automation  options  to  be  exercised, 

•  revise  the  estimates  of  workload  for  each  task,  and 

•  exercise  the  models  to  produce  new  workload 
predictions . 

The  automation  options  were  selected  in  consultation 
with  SMEs  from  the  LHX  office  within  the  Directorate  of 
Combat  Developments  (DCD)  at  the  U.  S.  Army  Aviation  Center 
(USAAVNC) ,  Fort  Rucker,  Alabama.  The  tasks  were  then 
reviewed  to  determine  how  each  proposed  automation  option 
would  change  the  workload  estimated  in  the  baseline  analysis. 
For  each  task  affected  by  automation,  new  descriptors  of 
workload  were  written.  These  descriptors,  in  turn,  were 
matched  with  the  verbal  anchors  to  assign  new  ratings  to  the 
workload  components.  Finally,  the  models  were  exercised  with 
the  revised  ratings  to  predict  workload  for  each  automation 
option  and  the  26  combinations  of  automation  options.  The 
results  of  these  exercises  were  used  to  evaluate  the  changes 
in  pilot  workload  that  would  occur  under  the  different 
automation  and  crewmember  configurations. 


Need 

Neither  the  workload  parameters  used  to  develop  the 
models  nor  the  workload  predictions  yielded  by  the  models 
have  been  validated.  Consequently,  this  research  project 
will  consist  of  (a)  a  validation  of  the  workload  parameters 
used  to  develop  the  models,  and  (b)  a  validation  of  the 
workload  predictions  yielded  by  the  models. 

Workload  parameters  used  to  develop  the  models  that 
require  validation  include  the: 

•  workload  ratings  assigned  to  each  task, 

•  total  workload  estimates  for  concurrent  tasks, 

•  estimated  time  required  to  perform  each  task, 

•  threshold  for  excessive  workload, 

•  temporal  relationships  among  tasks,  and 

•  procedural  relationships  among  tasks. 

Specific  predictions  yielded  by  the  models  that  require 
validation  include  four  indexes  of  excessive  workload  as 
listed  below: 

•  overloads, 

•  overload  conditions, 

•  overload  density,  and 

•  subsystem  overloads. 


The  specific  objectives  of  this  project  are  divided  into 
three  phases  of  research.  The  objective  of  Phase  1  is  to 
evaluate  the  reliability  of  (a)  the  workload  rating  scales 
used  to  rate  the  workload  components  of  each  operator  task, 
and  (b)  the  estimates  of  the  other  workload  parameters  used 
to  develop  the  LHX  workload  prediction  model.  The  objective 
of  Phase  2  is  to  obtain  validation  data  on  the  workload 
predicted  by  the  models  through  a  series  of  studies  employing 
part-mission  and  full-mission  flight  simulation.  The  objec¬ 
tive  of  Phase  3  is  to  refine  the  workload  prediction  model 
using  the  results  of  the  reliability  and  validation  research. 


Methodology 

A  research  plan  designed  to  meet  the  above  objectives 
was  produced  early  in  this  first  contract  year  (Aldrich  & 
Szabo,  1986) .  The  research  plan  provides  detailed  descrip¬ 
tions  of  18  tasks  required  to  accomplish  the  three  phases  of 
the  validation  research.  A  summary  of  the  research  method¬ 
ology  for  each  of  the  three  phases  is  described  below. 


Pha.SQ  1, 

Phase  1  consists  of  two  surveys  administered  to  human 
factors  scientists  who  are  SMEs  in  workload  research.  In  the 
first  survey,  all  combinations  of  paired  comparisons  of  the 
verbal  anchors  for  each  workload  rating  scale  are  presented 
to  the  SMEs.  The  SMEs  must  choose  which  anchor  in  each  pair 
imposes  more  attentional  demand.  The  results  of  this  survey 
will  assess  the  degree  of  agreement  among  the  SMEs  on  the 
workload  scales.  The  data  also  will  be  used  to  produce 
equal-interval  scales  (Engen,  1971)  to  replace  the  ordinal 
scales  that  were  used  in  the  original  workload  analysis. 

The  second  survey  asks  the  same  SMEs  to  use  the  work¬ 
load  scales  to  rate  the  short  descriptors  of  visual,  audi¬ 
tory,  cognitive,  and  psychomotor  workload  for  each  task  in 
the  model.  Correlational  techniques  will  be  used  to  evaluate 
the  interrater  reliability  of  the  workload  ratings. 


Phase  2 

Phase  2  consists  of  part-mission  and  full-mission 
simulation  experiments  to  validate  the  workload  estimates. 
For  the  part-mission  simulation,  mini-scenarios  will  be 


generated  by  selecting  concurrent  and  sequential  tasks  from 
the  mission  and  task  analysis.  For  the  full-mission  simu¬ 
lation,  a  composite  mission  scenario  will  be  developed  by 
selecting  segments  from  the  mission  and  task  analysis. 

The  part-mission  simulation  will  be  conducted  using  a 
repeated  measures  experimental  design  in  which  each  subject 
will  fly  the  mini-scenarios  multiple  times.  Results  will  be 
analyzed  to  assess  the  correlation  between  the  workload  model 
predictions  and  measures  of  the  operator's  performance  on  the 
concurrent  and  sequential  tasks.  The  correlation  coeffi¬ 
cients  will  serve  as  the  primary  measure  of  how  accurately 
the  workload  predictions  forecast  excessive  workload  at  the 
task  level  of  specificity.  To  assess  the  validity  of  the 
time  estimates  used  in  the  model,  the  actual  amount  of  time 
required  to  perform  the  various  tasks  in  the  mini-scenarios 
will  be  compared  with  the  times  estimated  during  the  task 
analysis.  The  procedural  relationships  among  the  tasks  will 
be  evaluated  by  noting  the  subjects'  ability  to  progress 
through  the  mini-scenarios  following  the  sequence  of  tasks 
specified  by  the  model. 

During  the  full-mission  simulation  experiments,  each 
trial  will  start  at  the  beginning  of  a  composite  scenario  and 
continue  without  interruption  to  the  end.  Analysis  of 
results  from  the  full-mission  simulation  will  include  all  of 
the  analyses  performed  during  the  part-mission  simulation 
data  analysis.  In  addition,  an  analysis  will  be  performed  to 
assess  the  effects  of  inserting  secondary  tasks  into  the 
composite  mission  scenario. 

The  final  task  in  Phase  2  will  be  to  compare  the  results 
from  the  part-mission  simulation  research  with  results  from 
the  full-mission  simulation  research.  Results  in  which 
workload  is  excessive  during  full-mission  simulation,  but  is 
not  excessive  when  the  same  concurrent  tasks  occur  in  part- 
mission  simulation,  will  indicate  whether  the  excessive 
workload  results  from  the  cumulative  effects  of  high  workload 
over  longer  periods  of  performance . 


Phase  3  consists  of  the  refinement  of  the  workload 
prediction  model  based  on  the  results  of  the  first  two 
phases.  The  first  refinements  will  occur  as  the  workload 
component  rating  scales  are  converted  from  ordinal  to 
interval  scales.  Further  refinements  will  be  made  to  the 
workload  model  algorithms  to  reflect  the  empirical  results  of 
the  part-mission  and  full-mission  simulation. 
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Results 


Phase  1 


The  paired  comparison  data  from  30  completed  surveys 
were  used  to  develop  each  rater's  rank  order  judgments  of  the 
verbal  anchor  atLentional  demand  requirements  for  the  four 
workload  component  scales.  The  rank  ordered  judgments  were 
analyzed  using  Kendall's  Coefficient  of  Concordance  to  assess 
the  degree  of  agreement  among  the  SMEs.  The  Coefficients  of 
Concordance  for  the  four  scales  are  as  follow: 

•  Visual  -  .40, 

•  Auditory  -  .49, 

•  Cognitive  -  .71,  and 

•  Psychomotor  -  .49. 


Phase  2 

The  initial  task  during  Phase  2  is  the  selection  of  a 
flight  simulation  facility  capable  of  supporting  the  part- 
mission  and  full-mission  simulation  studies  required  to 
validate  the  model.  Tentatively,  the  new  Crew  Station 
Research  and  Development  Facility  (CSRDF)  to  be  located  at 
the  Army's  Aerof light dynamics  Laboratory,  NASA  Ames, 
California,  was  selected  as  the  most  appropriate  site.  A 
final  selection  cannot  be  made  until  the  CSRDF  research 
flight  simulator  is  fabricated  by  the  contractor  (CAE 
Limited,  Montreal,  Canada) ,  and  is  installed  and  operational 
at  NASA  Ames.  Assurances  that  the  simulation  hardware 
(especially  the  visual  system)  and  software  will  be  adequate 
to  perform  the  necessary  research  are  required  before  a  final 
decision  can  be  made  to  utilize  the  CSRDF. 
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To  evaluate  the  potential  of  the  CSRDF  to  support  the 
mission  simulation  experiments,  Anacapa  researchers  conducted 
task  analyses  of  three  LHX  mission  scenarios  using  the 
specific  CAE  configuration  for  the  two-crewmember  simulation. 
The  Anacapa  analyses  identified  126  unique  crew  functions 
that  appear  in  75  different  mission  segments  from  the  three 
mission  scenarios.  Subsequently,  Anacapa  researchers  identi¬ 
fied  513  unique  tasks  in  the  126  functions  for  the  two- 
crewmember  configuration.  The  task  analysis  was  performed  at 
the  CAE  plant  in  Montreal  during  the  Army's  acceptance 
testing  of  the  simulation  hardware  and  software.  A^a'-  =  p=> 
researchers  verified  the  mission  analyses  by  personally 
performing  the  identified  tasks  in  the  flight  simulator  and 
by  presenting  the  tasks  and  workload  descriptors  for  review 
to  (a)  CSRDF  and  CAE  SMEs,  and  (b)  Army  pilots  serving  as 
subjects  during  the  simulator  acceptance  testing.  Draft 


working  papers  of  a  two-crewmember  task  analysis  were 
produced  that  include  workload  descriptors  and  subsystem 
descriptors  for  each  of  the  513  two-crewmember  tasks. 


Project  Status 

Phase  1 


The  next  step  in  Phase  1  will  be  to  use  the  pair 
comparison  data  to  produce  interval  scales  for  each  of  the 
workload  component  rating  scales.  Then,  the  second  survey 
described  in  the  Phase  1  methodology  section  will  be  designed 
and  administered  to  SMEs . 


Phase  2 

A  final  decision  to  use  the  CSRDF  to  conduct  the  flight 
simulation  research  is  required  before  the  Phase  2  research 
tasks  can  be  scheduled.  Meanwhile,  work  is  continuing  on  the 
task  analysis  being  conducted  for  the  CSRDF  two-crewmember 
conf igurat ion . 


Phase  3 

Refinement  of  the  workload  prediction  model  will  start 
as  soon  as  the  interval  scale  values  are  produced  during 
Phase  1.  The  new  workload  rating  values  for  each  of  the 
verbal  anchors  will  replace  the  ordinal  scale  values 
presently  in  the  task  data  base.  The  model  will  be  exercised 
to  produce  refined  workload  predictions  based  upon  the  new 
scale  values. 

As  the  part-mission  and  full-mission  simulation  results 
are  analyzed  during  Phase  2,  additional  refinements  will  be 
made  to  the  workload  prediction  model.  The  researchers  will 
make  necessary  corrections  to  the  workload  estimates,  time 
estimates,  and  decision  rules.  Refined  workload  predictions 
will  be  produced  using  the  empirically  derived  workload 
estimates,  time  values  and  updated  decision  rules,  computer 
files,  and  computer  programs. 
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DEVELOPMENT  AND  VALIDATION  OF  THE  AH- 64 
WORKLOAD  PREDICTION  MODEL 

Dr.  Sandra  M.  Szabo,  Project  Director 


The  Air/Land  Battle  2000  scenario  represents  a  high- 
threat  environment  that  will  place  heavy  workload  demands  on 
combat  helicopter  operators.  Advanced  technology  in  the 
latest  generation  of  Army  helicopters  is  designed  to  reduce 
crew  workload;  however,  in  some  instances,  the  tasks  required 
to  use  the  technology  have  actually  increased  workload.  The 
high  workload,  in  turn,  reduces  mission  effectiveness, 
increases  system  manning  and  personnel  requirements,  and 
increases  the  training  necessary  for  acquiring  and  main¬ 
taining  flight  proficiency. 

One  of  the  primary  reasons  that  technology  has  failed  to 
reduce  operator  workload  in  current  Army  aircraft  is  that  human 
factors  concepts  have  not  been  adequately  considered  during  the 
early  stages  of  system  design.  In  the  past,  no  methodology 
existed  for  assessing  the  workload  demands  of  emerging  weapon 
systems  prior  to  their  development .  Recently,  however,  the 
U.S.  Army  Research  Institute  Aviation  Research  and  Development 
Activity  (ARIARDA)  developed  a  methodology  for  evaluating  the 
role  of  advanced  technology  in  the  development  of  the  proposed 
light  helicopter  family  ( LHX)  aircraft.  The  methodology  for 
predicting  LHX  workload  in  advance  of  system  development  can  be 
extended  to  other  weapon  systems. 

As  part  of  its  aviation  Product  Improvement  Program 
(PIP),  the  Army  currently  is  developing  an  AH-64B  helicopter. 
The  Aviation  Systems  Command  (AVSCOM)  has  requested  that 
ARIARDA  adapt  the  LHX  workload  prediction  methodology  to  the 
AH-64A  system  to  assist  in  determining  the  impact  that  pro¬ 
posed  modifications  for  the  AH-64B  model  will  have  on  opera¬ 
tor  workload.  The  methodology  provides  a  mechanism  for 
avoiding  human  factors  errors  that  previously  have  resulted 
in  costly  production  changes  and  decreased  operator 
performance . 

In  response  to  AVSCOM' s  request  for  research  support, 
ARIARDA  has  conducted  a  comprehensive  task  analysis  of  the 
existing  AH-64A  mission.  The  task  analysis  data  are  cur¬ 
rently  being  used  to  develop  a  computer  model  that  can  be 
exercised  to  yield  predictions  of  workload  associated  with 
various  configurations  proposed  for  the  emerging  AH-64B 
helicopter . 


The  primary  objective  of  the  AH-64  workload  prediction 
research  is  to  determine  the  impact  that  advanced  technology 
is  likely  to  have  on  the  workload  of  AH-64  crewmembers. 
Specifically,  the  research  is  designed  to: 

•  identify  the  AH-64A  mission  functions  and  subsystems 
for  which  design  modifications  will  be  most  bene¬ 
ficial,  and 

•  determine  tne  predicted  impact  that  various  design 
modif ications  will  have  on  AH-64B  crew  workload. 

The  systematic  predictions  of  workload  yielded  by  the 
methodology  will  provide  valuable  input  into  early  human 
engineering  design  decisions  for  the  AH-64B  helicopter. 


The  research  approach  for  meeting  the  objectives 
consists  of  four  phases.  In  Phase  1,  the  AH-64A  helicopter 
was  used  to  conduct  a  comprehensive  mission/task/workload 
analysis  for  AH-64  crewmembers.  In  Phase  2,  data  provided  by 
the  mission/task/workload  analysis  are  being  used  to  develop 
a  baseline  computer  model  for  predicting  AH-64A  crew  work¬ 
load.  In  Phase  3,  the  computer  model  will  be  exercised  to 
predict  the  impact  that  design  modifications  are  likely  to 
have  on  crew  workload  for  various  AH-64B  configurations.  In 
Phase  4,  the  workload  predictions  yielded  by  the  model  will 
be  validated.  Each  of  the  phases  is  described  below. 


Phase  1  of  the  research  consists  of  a  comprehensive 
task/workload  analysis  of  all  phases  of  the  AH-64A  attack 
mission.  The  research  tasks  performed  during  the  conduct  of 
the  task  analysis  are  listed  below: 

•  develop  a  composite  mission  scenario, 

•  divide  the  composite  scenario  into  mission  phases, 

•  divide  the  mission  phases  into  segments, 

•  identify  the  functions  in  the  mission  segments, 

•  identify  the  tasks  for  each  function, 

•  identify  the  subsystem(s)  associated  with  each  task, 

•  identify  the  crewmember  performing  each  task, 

•  estimate  the  workload  for  each  task,  and 

•  estimate  the  time  required  to  perform  each  task. 

Each  of  the  research  tasks  is  briefly  summarized  in  the 
sections  that  follow. 
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Develop  a  composite  mission  scenario.  Five  AH-64A 
attack  mission  profiles,  prepared  by  the  Directorate  of 
Combat  Developments  (DCD)  at  the  U.S.  Army  Aviation  Center 
(USAAVNC) ,  were  examined.  The  profiles  were  used  to  develop 
a  composite  scenario  for  the  AH-64A  attack  mission.  In  the 
scenario,  the  mission  begins  with  preflight  and  departure 
operations  performed  in  the  assembly  area  ;  the  pilot  flies 
from  the  assembly  area  to  the  holding  area,  where  mission 
coordination  is  conducted.  From  the  holding  area,  the  pilot 
flies  to  the  battle  area  where  target  acquisition  and  engage¬ 
ment  functions  occur.  When  the  weapons  are  expended,  the 
pilot  flies  to  the  forward  area  arming  and  refueling  point 
(FARP)  to  refuel  and  rearm  the  aircraft.  Following  refueling 
and  rearming,  the  crew  returns  to  the  battle  area  and  again 
expends  the  weapons.  The  pilot  then  returns,  via  the  holding 
area,  to  the  assembly  area,  where  terminal  operations  and 
postflight  procedures  are  performed. 


Other  assumptions  characterized  by  the  scenario  include 
the  following: 

•  the  pilot's  primary  function  is  to  fly  the  aircraft 
and  the  gunner's  primary  function  is  to  acquire  and 
engage  targets, 

•  optimal  flight  conditions  <e.g.,  excellent  weather 
conditions,  no  aircraft  emergencies)  exist  throughout 
the  mission,  and 

•  all  reconnaissance  and  team  leader  functions  are 
performed  by  the  scout  pilot. 


Divide  the  scenario  into  mission  phases.  The  composite 
scenario  was  divided  into  seven  mission  phases.  The  phases 


Preflight , 

Departure , 

Enroute, 

Target  Servicing, 

FARP  Operations, 

Terminal  Operations,  and 
Post  f 1 ight . 


Divide  the  mission  phases  into  segments.  Each  of  the 
seven  mission  phases  was  subsequently  divided  into  mission 
segments.  A  mission  segment  is  defined  as  a  major  group  of 
events  that  have  a  definite  start  and  end  point  during  a 
mission  phase.  The  numbei  of  segments  identified  in  each 
phase  is  listed  below: 

•  Preflight  -  6, 

•  Departure  -  2, 

•  Enroute  -  8, 
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•  Target  Servicing  -  33, 

•  FARP  Operations  -  4, 

•  Terminal  Operations  -  2,  and 

•  Postflight  -  2. 

Of  the  57  segments  occurring  throughout  the  mission,  52 
segments  are  unique  (i.e.,  segments  that  are  distinctly 
different  from  all  other  segments) . 

Identify  the  functions  in  each  segment.  Each  of  the  52 
unique  segments  was  further  divided  into  functions.  A  func¬ 
tion  is  defined  as  a  set  of  activities  that  must  be  performed 
either  by  an  operator  or  by  equipment  to  complete  a  portion 
of  the  mission  segment.  A  total  of  159  unique  functions  was 
identified  for  the  segments. 

Identify  the  tasks  for  each  function.  Each  of  the  159 
unique  functions  was  further  divided  into  tasks  considered 
critical  to  the  successful  performance  of  the  function.  Each 
task  consists  of  a  verb  and  an  object;  the  verb  describes  the 
action,  and  the  object  describes  the  recipient  of  the  action. 
The  tasks  are  the  basic  units  of  the  task  analysis.  A  total 
of  688  unique  tasks  was  identified  for  the  mission  functions. 

Identify  the  subsystem (s)  associated  with  each  task. 

For  each  unique  task,  the  subsystem  associated  with  the  per¬ 
formance  of  the  task  was  identified.  The  subsystems  repre¬ 
sent  the  following  major  categories  of  equipment: 

•  Armament, 

•  Flight  Control, 

•  Utility, 

•  Engine, 

•  Navigation, 

•  Safety,  and 

•  Visual . 

Identify  the  crewmember  performing  each  task.  The  next 
task  in  the  analysis  was  to  identify  the  crewmember  who 
performs  each  task.  Using  AH-64A  checklists,  operator's 
manuals,  flight  training  guides,  and  subject  matter  experts 
(SMEs)  ,  the  researchers  assigned  each  task  to  the  pilot, 
gunner,  or  both  crewmembers.  In  general,  the  pilot  was 
assigned  all  flight  tasks,  and  the  gunner  was  assigned  target 
acquisition  and  engagement  tasks. 


Estimate  the  workload  required  for  each  task.  Workload, 
as  used  in  the  present  analysis,  consists  of  three  compo¬ 
nents:  sensory,  cognitive,  and  psychomotor.  The  sensory 

component  refers  to  the  complexity  of  the  visual,  auditory, 
and  kinesthetic  stimuli  to  which  the  operator  must  respond; 


the  cognitive  component  refers  to  the  level  of  thinking 
required;  and  the  psychomotor  component  refers  to  the  com¬ 
plexity  of  the  behavioral  response  required.  A  short  verbal 
descriptor  of  each  of  the  workload  components  was  written  for 
each  task.  The  descriptors  were  then  compared  to  verbal 
anchors  contained  in  7-point  rating  scales  designed  to  mea¬ 
sure  workload  for  each  of  the  five  workload  components.  In 
each  instance,  a  consensus  was  reached  by  the  two  analysts 
who  had  initially  assigned  the  ratings  independently;  the 
consensual  ratings  were  subsequently  reviewed  by  three  AH-64A 
SMEs  . 

Estimate  the  time  to  perform  each  task.  To  develop  a 
timeline  for  the  AH-64A  mission,  it  was  necessary  to  derive 
estimates  of  the  time  required  to  perform  each  task.  Esti¬ 
mates  for  many  of  the  tasks  identified  in  the  mission  analy¬ 
sis  were  derived  by  timing  the  actual  performance  of  the 
tasks  in  the  Cockpit,  Weapons,  and  Emergency  Procedures 
Trainer  (CWEPT) .  Estimates  for  the  remaining  tasks  are  based 
on  the  subjective  judgments  of  SMEs. 

Prior  to  assigning  the  task  times,  each  task  was  cate¬ 
gorized  as  discrete  fixed,  discrete  random,  or  continuous. 

The  three  task  categories  are  defined  as  follows: 

•  discrete  fixed  tasks--tasks  that  have  definite  start  and 
end  points  in  the  function  (e.g.,  setting  switches); 

•  discrete  random  tasks — discrete  tasks  that  occur 
intermittently  or  randomly  during  a  portion  of  the 
function  (e.g.,  checking  flight  instruments);  and 

•  continuous  tasks--tasks  that  occur  contii  usly  during 
a  portion  of  the  function  (e.g.,  monitoring  audio) . 

Following  the  completion  of  the  task  analysis,  the  data 
were  summarized  on  function  analysis  worksheets.  The  func¬ 
tion  analysis  worksheets  were  subsequently  reviewed  by  three 
AH- 64 A  SMEs . 


Phase  2 

Phase  2  of  the  research  consists  of  the  development  of  a 
baseline  computer  model  to  predict  total  workload  for  AH-64A 
crewmembers.  The  principal  tasks  in  developing  the  model  are 
listed  below: 

•  establish  computer  files  for  the  mission/task /workload 
analysis  data, 

•  develop  function  and  segment  decision  rules, 


•  write  computer  programs  to  implement  the  decision 
rules,  and 

•  produce  computer  estimates  of  workload  for  the  base¬ 
line  AH-64A  configuration. 

Each  of  these  tasks  is  described  briefly  below. 

Establish  computer  files.  The  initial  task  in  Phase  2 
of  the  research  was  to  enter  the  mission/task/workload 
analysis  data,  derived  during  Phase  1,  into  a  computer  data 
base.  Information  reported  on  the  function  analysis  work¬ 
sheets  was  used  to  create  the  following  data  files: 

•  a  list  of  phases, 

•  a  list  of  segments, 

•  a  list  of  functions, 

•  a  list  of  tasks, 

•  crewmember  performing  each  task, 

•  estimates  of  workload  for  each  task, 

•  estimates  of  time  for  each  task, 

•  a  list  of  subsystem  identifiers,  and 

•  a  description  of  switches. 


mission/task  analysis  conducted  during  Phase  1  used  a  top- 
down  approach  in  which  the  tasks  were  identified  as  the  basic 
elements  of  the  mission.  The  computer  model  developed  during 
Phase  2  used  a  bottom-up  approach  in  which  the  tasKS  con¬ 
tained  in  the  computer  files  were  combined  to  form  functions 
which,  in  turn,  were  combined  to  form  segments. 

For  each  of  the  159  unique  functions  identified  in  the 
Phase  1  analysis,  a  function  summary  sheet  was  developed  to 
identify  the  specific  tasks  performed  by  each  crewmember. 
Function  decision  rules  were  then  written  to  identify  the 
sequence  and  time  for  the  performance  of  these  tasks. 
Following  the  development  of  the  function  summary  sheets  and 
decision  rules,  segment  summary  sheets  and  decision  rules 
were  written.  The  segment  decision  rules  specify  the  pro¬ 
cedure  (e.g.,  sequence  and  time)  for  combining  the  functions, 
created  by  the  function  decision  rules,  to  form  each  mission 
segment . 

Write  computer  programs .  The  time-based  function  and 
segment  decision  rules  are  the  blueprints  for  placing  the 
tasks  performed  by  each  crewmember  at  the  appropriate  point 
on  the  mission  timeline.  To  permit  an  automated  analysis  of 
workload,  computer  programs  will  be  developed  to  implement 
the  decision  rules.  The  computer  programs  will  provide  a 
means  for  identifying  all  the  tasks  performed  by  a  given 
crewmember  at  any  point  within  the  mission.  In  addition,  the 
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programs  will  specify  the  procedure  for  summing  the  workload 
ratings  for  concurrent  tasks  to  produce  estimates  of  total 
workload  for  each  crewmember. 


workload .  When  the  computer  programs  are  written,  the  model 
can  be  exercised  to  produce  estimates  of  total  workload 
associated  with  the  performance  of  concurrent,  as  well  as 
sequential  tasks  in  the  baseline  AH-64A  configuration.  The 
total  workload  for  concurrent  tasks  will  be  computed  by 
summing  the  ratings  assigned  during  the  task  analysis  to  each 
workload  component  (i.e.,  visual,  auditory,  kinesthetic, 
cognitive,  and  psychomotor) .  By  using  the  sums,  the  model 
will  identify  points  on  the  mission  timeline  at  which  exces¬ 
sive  workload,  referred  to  as  "overload,"  will  occur.  The 
four  indexes  of  overload  are  listed  and  defined  below. 

•  A  component  overload  occurs  whenever  the  sum  of  the 
ratings  assigned  to  a  given  workload  component  for 
concurrent  tasks  equals  "8"  or  higher.  Thus,  as  many 
as  five  component  overloads  can  occur  at  any  point  on 
the  mission  timeline. 

•  An  overload  condition  occurs  whenever  an  overload,  as 
defined  above,  occurs  in  at  least  one  component  of  two 
or  more  concurrent  tasks.  Thereafter,  an  overload 
condition  is  counted  any  time  a  change  in  the  tasks 
and/or  the  components  contributing  to  an  overload 
occurs . 

•  Overload  density  is  the  percentage  of  time  within  a 
mission  segment  during  which  a  component  overload 
occurs . 

•  Subsystem  overloads  are  computed  by  tallying  the 
number  of  times  each  subsystem  is  associated  with  an 
overload . 


During  Phase  3  of  the  research,  the  computer  model  exer¬ 
cised  in  Phase  2  to  provide  a  baseline  analysis  of  AH-64A 
crew  workload,  will  be  exercised  to  predict  how  much  crew 
workload  might  be  reduced  by  proposed  configurations  for  the 
AH-64B  aircraft.  The  methodology  consists  of  the  following 
steps  : 

•  identify  the  automation  options  proposed  for  the  AH- 
64 B  model, 

•  conduct  a  task/workload  analysis  for  each  option, 


•  exercise  the  model  to  yield  revised  estimates  of 
workload,  and 

•  conduct  a  comparability  analysis  of  the  estimates  of 
workload  for  the  baseline  and  automated 
configurations . 

The  results  of  Phase  3  can  be  used  to  provide  estimates  of 
the  potential  impact  of  the  proposed  automation  options  on 
the  workload  of  AH-64B  crewmembers.  These  estimates,  in 
turn,  will  assist  design  engineers  in  determining  the  optimal 
configuration  for  the  AH-64B  aircraft. 


Phase  4 

During  Phase  4,  the  workload  prediction  model  will  be 
validated.  Validation  of  the  model  will  consist  of  two  major 
steps : 

•  validation  of  the  workload  component  rating  scales, 
and 


•  validation  of  the  workload  predictions  yielded  by  the 
model . 

Validation  of  the  workload  component  rating  scales  will 
be  established  by  performing  the  following  tasks: 

•  determine  the  interrater  reliability  for  the  rank 
order  of  the  verbal  anchors  within  each  7-point  scale, 
and 

•  determine  the  interrater  reliability  for  assigning  the 
numerical  ratings  to  the  verbal  descriptors  of 
workload . 

Validation  of  the  workload  predictions  yielded  by  the 
model  will  be  established  by  conducting  part-mission  and 
full-mission  simulation  studies.  In  each  instance,  predic¬ 
tions  of  workload  for  specific  tasks  will  be  compared  with 
measures  of  primary  and  secondary  task  performance,  as  well 
as  other  subjective  measures  of  workload,  such  as  Subjective 
Workload  Assessment  Technique  (SWAT)  ratings  (Reid, 
Shingledecker ,  &  Eggemeier,  1981),  taken  during  mission 
simulation.  Correlation  coefficients  between  the  workload 
predictors  and  the  criterion  measures  of  workload  (e.g.,  task 
performance  and  SWAT  ratings)  will  be  used  to  assess  the 
validity  of  the  model  for  predicting  workload.  The  results 
of  the  validation  studies  of  both  the  rating  scales  and  the 
workload  predictions  will  subsequently  be  used  to  refine  the 
mode  1 . 


V 


The  workload  prediction  methodology  developed  by  ARIARDA 
provides  a  systematic  means  for  predicting  human  operator 
workload  in  advance  of  system  design.  The  systematic  predic¬ 
tions  of  workload,  in  turn,  provide  an  excellent  foundation 
for  human  engineering  decisions  early  in  the  development 
process  when  decisions  are  made  about  the  functions  that 
should  be  assigned  to  machines.  Thus,  the  model  provides  a 
decision-making  tool  that  reduces  costly  changes  associated 
with  PIPs. 

In  addition,  the  methodology  provides  information  for 
identifying  emerging  system  personnel  and  training  require¬ 
ments.  By  assisting  in  the  identification  of  these  require¬ 
ments,  the  methodology  provides  a  means  for  factoring  total 
system  costs  into  trade-off  studies  conducted  during  the 
early  stages  of  system  development .  Such  a  concept  repre¬ 
sents  a  dramatic  change  from  the  system  development  process 
used  in  the  past  . 


At  the  beginning  of  the  current  contract  period, a 
preliminary  mission/task/workload  analysis  had  been  completed 
and  preliminary  decision  rules  for  the  mission  functions  and 
segments  had  been  developed.  A  draft  technical  report 
entitled,  "A  Comprehensive  Task  Analysis  of  the  AH-64  Mission 
with  Crew  Workload  Estimates  and  Preliminary  Decision  Rules 
for  Developing  an  AH-64  Workload  Prediction  Model,"  (Szabo  & 
Bierbaum,  1986)  had  been  written  and  submitted  to  ARIARDA. 

The  report  describes  the  research  methodology  and  presents 
appendixes  containing  the  decision  rules  and  the  results  of 
the  task  analysis. 

During  the  first  year  of  the  current  contract,  both  the 
mission/task/workload  analysis  and  the  decision  rules  have 
undergone  extensive  review  and  revision.  In  addition,  a 
preliminary  version  of  the  computer  model,  using  a  Perkin- 
Elmer  minicomputer  and  FORTRAN  language,  was  developed.  The 
model  was  exercised  to  produce  a  preliminary  analysis  of 
workload  for  each  of  the  mission  segments. 


A  final  version  of  the  computer  model  will  be  developed 
using  a  Zenith  microcomputer  and  Turbo  PASCAL  software.  The 
model  will  then  be  exercised  to  produce  estimates  of  workload 
for  each  of  the  52  unique  mission  segments.  A  technical 
report  will  be  written  to  present  the  results  of  the  segment 
analyses  for  the  baseline  configuration.  As  system  design 
developments  for  the  AH-64B  aircraft  occur,  additional 
analyses  will  be  conducted  to  determine  the  predicted  impact 
of  the  proposed  design  configurations  on  crew  workload. 

Tentative  plans  have  been  made  by  ARIARDA  to  use  either 
the  AH-64  cmbat  mission  simulator  at  Fort  Rucker,  Alabama, 
or  the  AH-64B  combat  mission  simulator  at  McDonnell-Douglas 
in  Mesa,  Arizona,  to  validate  the  computer  model.  Assuming 
the  required  support  is  provided,  it  is  projected  that  (a) 
the  model  can  be  exercised  to  predict  the  impact  of  proposed 
AH-64B  modifications  in  mid-FY  1988,  and  (b)  the  model  can  be 
validated  in  FY  1989. 
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DEVELOPMENT  OF  A  WORKLOAD  PREDICTION  MODEL 
FOR  MH-60K  AND  MH-47E  HELICOPTER  MODIFICATIONS 

Mr.  Carl  R.  Bierbaum,  Project  Director 


Background 

The  Special  Operations  Forces  (SOF)  Aviation  Project 
Office  at  the  Army's  Aviation  Systems  Command  (AVSCOM)  has 
been  tasked  to  modify  existing  UH-60A  and  CH-47D  aircraft  for 
SOF  missions.  To  satisfy  the  tasking,  AVSCOM  is  engaged  in  a 
large-scale  modification  program  to  develop  the  MH-60K  and 
MH-47E  aircraft. 

The  high  technology  modifications  being  proposed  for  the 
MH-60K  and  the  MH-47E  may  increase  workload  by  placing  addi¬ 
tional  demands  on  the  mental  resources  of  the  crewmembers. 
Anacapa  Sciences,  Inc.,  personnel,  under  contract  to  the  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
(ARIARDA) ,  have  developed  a  methodology  for  predicting 
operator  workload  in  advance  of  system  design.  Initially, 
the  workload  prediction  methodology  was  developed  and  applied 
to  the  design  of  the  Army's  light  helicopter  family  (LHX) 
aircraft  (Aldrich  &  McCracken,  1984).  The  LHX  methodology 
was  refined  and  used  to  develop  a  baseline  model  that 
predicts  workload  encountered  by  operators  of  the  AH-64 
aircraft  (Szabo  &  Bierbaum,  1986) .  This  research  will  apply 
the  Anacapa /ARIARDA  workload  prediction  methodology  to  the 
development  of  a  model  that  will  yield  estimates  of  workload 
for  proposed  configurations  of  the  MH-60K  and  MH-47E 
helicopters . 


Need 

To  conduct  an  adequate  assessment  of  MH-60K  and  MH-47E 
crew  performance  and  workload,  a  mission/task/workload 
analysis  is  needed  for  the  SOF  scenario.  Since  ARIARDA  has 
developed  a  methodology  for  conducting  mission/task  analysis 
and  workload  prediction  (Aldrich,  Szabo,  &  Craddock,  1985), 
the  SOF  Aviation  Project  Office  requested  ARIARDA  to  provide 
them  with  mission/task  analyses  and  workload  predictions  for 
the  MH-60K  and  MH-47E  aircraft.  The  refined  methodology  used 
for  the  AH-64  mission/task  analysis  is  being  adapted  and  used 
to  conduct  the  required  mission/task  analyses  for  the  UH-60A 
and  CH-47D  and  to  yield  workload  predictions  for  the  modified 
aircraft  . 


The  objective  of  this  project  is  to  determine  the  impact 
that  high  technology  modifications  are  likely  to  have  on  the 
workload  of  UH-60A  and  CH-47D  crewmembers.  Specifically,  the 
research  is  designed  to: 

•  provide  a  mission/task  analysis  of  the  UH-60A  in  its 
present  mission  profile  to  form  a  baseline  for  deter¬ 
mining  the  impact  that  the  proposed  MH-60K  modifica¬ 
tions  will  have  on  crew  workload,  and 

•  provide  a  mission/task  analysis  of  the  CH-47D  in  its 
present  mission  profile  to  form  a  baseline  for  deter¬ 
mining  the  impact  that  the  proposed  MH-47E  modifica¬ 
tions  will  have  on  crew  workload. 


The  research  approach  adapted  for  meeting  the  objectives 
of  this  project  is  a  refinement  of  the  research  conducted  for 
the  AH-64A  (Szabo  &  Bierbaum,  1986) .  The  research  consists 
of  six  phases . 

•  Phase  1  is  a  baseline  mission/task  analysis  of  crew 
workload  for  the  UH-60A  helicopter. 

•  Phase  2  is  the  development  of  a  computer  model  of  UH- 
60A  workload  utilizing  the  data  provided  by  the 
mission/task  analysis  conducted  in  Phase  1. 

•  Phase  3  is  a  baseline  mission/task  analysis  of  crew 
workload  for  the  CH-47D  helicopter. 

•  Phase  4  is  the  development  of  a  computer  model  of  CH- 
47D  workload  utilizing  the  data  provided  by  the 
mission/task  analysis  conducted  in  Phase  3. 

•  Phase  5  is  the  exercise  of  the  computer  model  to 
predict  the  impact  that  design  modifications  are 
likely  to  have  on  crew  workload  for  various  MH-6QK 
configurations . 

•  Phase  6  is  the  exercise  of  the  computer  model  to 
predict  the  impact  that  design  modif ications  are 
likely  to  have  on  crew  workload  for  various  MH-47E 
configurations . 

The  methodology  for  each  of  the  six  phases  is  described 
belov  ■ . 
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Phase  1 


Phase  1  consists  of  a  mission/task  analysis  of  the  UH- 
60A  support  mission.  The  tasks  performed  for  the  task 
analysis  are  listed  below: 

•  develop  a  mission  scenario, 

•  divide  the  mission  scenario  into  mission  phases, 

•  divide  the  mission  phases  into  segments, 

•  divide  the  segments  into  functions, 

•  identify  the  specific  tasks  for  each  function, 

•  identify  the  subsystem  associated  with  each  task, 

•  estimate  the  workload  for  each  task,  and 

•  estimate  the  time  required  to  perform  each  task. 

A  brief  summary  of  each  task  is  described  in  succeeding 
paragraphs . 

Develop  a  mission  scenario.  The  UH-60A  mission  scenario 
was  developed  by  reviewing  the  Army  Training  and  Education 
Programs  (ARTEPs)  and  interviewing  UH-60A  subject  matter 
experts  (SMEs) .  The  basic  mission  of  the  UH-60A  aircraft  is 
to  provide  air  transportation  of  personnel  and  cargo  in 
support  of  combat  operations.  The  mission  can  be  conducted 
either  day  or  night,  and  may  include  both  internal  loads  and 
external  sling  leads. 


The  following  assumptions  were  made  during  development 
of  the  mission  scenario: 

•  the  pilot's  primary  role  is  to  fly  the  aircraft,  while 
the  copilot's  role  is  to  assist  the  pilot  and  perform 
navigation  functions;  and 

•  the  mission  is  flown  under  optimal  day  or  night  condi¬ 
tions  (i.e.,  full  moon  and  no  degradation  due  to 
weather  or  emergencies)  . 


Divide  the  mission  scenario  into  mission  phases.  After 
the  mission  scenario  is  developed,  the  mission  is  divided 
into  operational  phases.  The  phases  are  major  steps  in  the 
mission  where  similar  operations  and  activities  are  per¬ 
formed.  The  beginning  and  end  of  phases  are  identified  by 
configuration  changes  in  the  aircraft  and  new  sets  of  activi¬ 
ties  by  the  crew.  Groupings  of  activities  and  events 
associated  with  major  mission  segments,  such  as  departure, 
enroute,  and  arrival  constitute  mission  phases. 


Divide  the  mission  phases  into  segments.  The  mission 
phases  selected  for  analysis  are  divided  into  segments.  A 
mission  segment  is  defined  as  a  major  sequence  of  events  that 
has  a  definite  start  and  end  point  during  a  mission  phase. 
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Divide  the  segments  into  functions.  A  function  is 
defined  as  a  set  of  activities  that  must  be  performed  by  an 
operator  to  complete  a  portion  of  the  segment.  After  all 
functions  are  identified,  the  unique  functions  are  listed 
alphabetically  in  a  computer  data  file.  The  ordinal  position 
of  each  function  within  the  alphabetical  list  is  used  to 
assign  a  numerical  identif ication  code  to  each  function. 

Identify  the  specific  tasks  for  each  function.  Each 
function  is  further  divided  into  tasks.  Each  task  defines  a 
specific  crew  activity  that  is  essential  to  the  successful 
performance  of  the  functions.  The  task  description  consists 
of  a  verb  and  an  object;  the  verb  describes  the  action  by  the 
crewmember  and  the  object  describes  the  recipient  of  the 
action.  The  list  of  unique  tasks  are  alphabetized  by  object 
and  assigned  a  numerical  identification  code.  The  tasks  a'-Q 
the  basic  unit  of  the  mission  analysis. 

Identify  the  subsystem  associated  with  each  task.  The 
subsystem  associated  with  each  task  is  identified.  The 
identification  of  tasks  with  their  respective  subsystems 
facilitates  future  analysis  of  automation  options  and  other 
aircraft  system  modifications. 

Estimate  the  workload  for  each  task.  Workload,  as  the 
term  is  used  in  this  research,  is  defined  as  the  attentional 
demand  placed  on  the  operator (s)  as  they  perform  the  mission 
tasks.  The  research  methodology  separately  treats  three 
components  of  workload:  sensory,  cognitive,  and  psychomotor. 
The  sensory  component  refers  to  the  complexity  of  the  visual 
unaided,  visual  aided, *  auditory,  and  kinesthetic  stimuli  to 
which  an  operator  must  attend;  the  cognitive  component  refers 
to  the  level  of  information  processing  required  from  the 
operator;  and  the  psychomotor  component  refers  to  the  com¬ 
plexity  of  the  operator's  behavioral  responses. 

To  determine  the  workload,  short  verbal  descriptors  are 
written  for  each  task.  These  descriptors  are  then  matched 
with  the  verbal  anchors  on  the  workload  component  rating 
scales  (Szabo  &  Bierbaum,  1986) . 

Estimate  the  time  required  to  perform  each  task .  To 
develop  a  timeline,  each  task  is  first  identified  as  a 
discrete  fixed,  discrete  random,  or  continuous  task.  The 
three  categories  of  tasks  are  defined  as  follow: 


*Visual  unaided  refers  to  visual  acuity  with  the  naked  eye. 
Visual  aided  is  the  visual  acuity  with  the  use  of  night 
vision  goggles  or  other  night  vision  device; . 


•  discrete  fixed--tasks  that  have  definite  start  and  end 
points  within  the  function, 

•  discrete  random--discrete  tasks  that  occur  inter¬ 
mittently  or  randomly  during  a  portion  of  the 
function,  and 

•  continuous--tasks  that  occur  continuously  during  a 
portion  of  the  function. 

Time  is  then  assigned  to  each  task  by  observations  of  perfor¬ 
mance  time  in  simulators  or  by  estimates  provided  by  SMEs  who 
have  experience  performing  the  tasks. 


The  development  of  the  computer  model  for  the  workload 
in  Phase  2  consists  of  the  following  tasks: 

•  develop  preliminary  decision  rules, 

•  establish  a  series  of  data  files  from  the  information 
derived  through  the  mission/task/workload  analysis. 

•  develop  computer  programs  to  implement  the  decision 
rules, 

•  produce  predicted  workload  for  the  baseline  UH-60A 
configuration . 

Each  of  these  tasks  is  described  briefly  below. 

Development  of  preliminary  decision  rules-  The  mission/ 
task/workload  analysis  performed  in  Phase  1  uses  a  top-down 
approach  to  identify  the  tasks  that  must  be  performed.  That 
is,  the  analysis  starts  with  the  mission  and  proceeds,  top- 
down,  through  the  phases,  segments,  and  functions  to  the  task 
level.  In  developing  the  crewmember  workload  model,  a 
bottom-up  approach  is  used.  The  specific  tasks  identified 
during  the  analysis  are  combined  to  form  the  unique  func¬ 
tions;  the  functions  are  then  combined  to  form  the  mission 
segments.  Preliminary  decision  rules  are  written  for  com¬ 
bining  (a)  the  tasks  into  functions,  and  (b)  the  functions 
into  segments.  The  decision  rules  are  designed  to  represent 
the  expected  behavior  of  the  crewmembers  as  they  perform 
mission  tasks  at  each  half-second  interval  on  a  timeline.  By 
summing  the  estimates  of  workload  associated  with  concurrent 
tasks  within  concurrent  functions,  estimates  of  total  work¬ 
load  can  be  derived  at  each  half-second  interval  for  each 
c  rewmember . 


Establish  data  files.  A  series  of  data  files  for  the 
information  derived  through  the  mi ssion/task/workload 


analysis  is  developed.  The  following  specific  files  are 
established : 

•  a  list  of  mission  segments, 

•  a  list  of  unique  functions, 

•  a  list  of  unique  tasks, 

•  a  list  of  subsystem  identifiers, 

•  estimates  of  the  sensory,  cognitive,  and  psychomotor 
workload  for  each  task, 

•  estimates  of  the  duration  of  each  task,  and 

•  a  list  of  types  of  switches. 

Information  contained  in  the  files  serve  as  the  data  base  for 
developing  the  UH-60A  crew  workload  estimation  model. 


Produce  predicted  workload  for  the  baseline  configura¬ 
tion  .  The  total  workload  for  concurrent  tasks  is  computed  by 
summing  the  ratings  assigned  during  the  task  analysis  to  each 
workload  component  (i.e.,  visual,  auditory,  kinesthetic, 
cognitive,  and  psychomotor)  for  each  concurrent  task.  By 
using  these  sums,  the  model  identifies  points  on  the  timeline 
at  which  the  performance  of  concurrent  tasks  results  in 
predictions  of  excessive  workload. 


Piaase  .1.  and  Ehase  1 

The  research  approach  adopted  for  the  fu-,r 70  during 
Phases  3  and  4  is  identical  to  that  used  for  the  UH-60A  in 
Phases  1  and  2  except  for  the  differences  in  aircraft. 


Phase  5 

During  Phase  5,  the  computer  baseline  model  will  be  used 
to  predict  workload  associated  with  the  proposed  configura¬ 
tions  for  the  MH-60K  aircraft.  To  accomplish  this,  new  work¬ 
load  descriptors  will  be  written  for  each  of  the  mission 
tasks  that  will  be  affected  by  proposed  modifications.  New 
workload  estimates  wilJ  be  generated  by  comparing  the  new 
descriptors  to  the  verbal  anchors  on  the  workload  component 
scales.  Computer  files  will  be  established  with  the  new 
information,  and  the  model  will  be  exercised  to  predict  the 
impact  that  the  proposed  automation  options  are  likely  to 
have  on  pilot  and/or  copilot  workload. 


Once  the  computer  baseline  model  has  been  developed  for 
the  CH-47D,  the  research  approach  described  in  Phase  5  will 
be  used  to  predict  workload  associated  with  the  proposed 
configurations  for  the  MH-47E  helicopter. 


Project  Status 

Work  Completed 

Phase  1  of  this  research  started  in  December  1986.  Each 
of  the  completed  tasks  is  described  briefly  below. 

Develop  a  mission  scenario.  The  UH-60A  mission  begins 
in  an  assembly  area  (AA) .  The  pilot  flies  contour  flight 
from  the  AA  to  a  pick-up  zone  (PZ)  where  cargo  and/or  troops 
are  assembled  for  pick  up.  After  completing  loading 
operations,  the  pilot  flies  nap-of-the-earth  (NOE)  to  the 
landing  zone  (LZ)  to  insert  the  combat  troops  or  deliver  the 
cargo.  After  completing  the  delivery,  the  pilot  flies  NOE 
back  to  the  PZ  for  another  load.  This  pattern  is  continued 
until  refueling  is  required.  The  pilot  then  flies  NOE  from 
the  LZ  to  the  forward  area  arming  and  refueling  point  (FARP) , 
where  refueling  operations  are  conducted.  Upon  completion  of 
the  FARP  operations,  the  crew  returns  to  the  PZ  for  continua¬ 
tion  of  the  mission.  When  the  mission  is  complete,  the  pilot 
flies  contour  back  to  the  AA.  Preflight  and  postflight 
activities  are  not  included  in  the  analysis. 

Divide  the  mission  scenario  into  mission  phases.  The 
mission  scenario  was  divided  into  nine  phases  as  follow: 

•  Departure  (Assembly  Area)  , 

•  Enroute  (AA-PZ) , 

•  Departure  (PZ)  , 

•  Enroute  (PZ-LZ) , 

•  Departure  (LZ), 

•  Enroute  (LZ-PZ)  .  :  (LZ-FARP), 

•  FARP  Operations, 

•  Enroute  (FARP-PZ) ,  and 

•  Enroute  (PZ-AA) . 

Divide  the  mission  phases  into  segments.  The  nine 
mission  phases  were  divided  into  34  unique  segments.  Some 
segments  are  used  in  more  than  one  of  the  phases.  The  number 
of  segments  in  each  phase  is  listed  below. 

•  Departure  (Assembly  Area)  -  3, 

•  Enroute  (AA-PZ)  -  10, 
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•  Departure  (PZ)  -  7, 

•  Enroute  (PZ-LZ)  -  12, 

•  Departure  (LZ)  -  3, 

•  Enroute  (LZ-PZ)  or  (LZ-FARP)  -  10, 

•  FARP  Operations  -  5, 

•  Enroute  (FARP-PZ)  -  10,  and 

•  Enroute  (PZ-AA)  -  10. 

Divide  the  segments  into  functions.  Each  of  the  34 
unique  segments  was  divided  into  functions.  A  total  of  48 
unique  functions  was  identified,  alphabetized,  and  assigned  a 
numerical  identification  code. 

Identify  the  specific  tasks  for  each  function.  Each  of 
the  48  functions  was  divided  into  tasks.  A  total  of  138 
unique  tasks  was  identified,  alphabetized  by  object,  and 
assigned  a  numerical  identification  code. 

Identify  the  subsystem  associated  with  each  task . 
Seventeen  subsystems  were  identified  under  five  major  cate¬ 
gories.  The  major  categories  are  listed  below: 

•  Engine, 

•  Flight  Control, 

•  Navigation, 

•  Utility,  and 

•  Visual. 

Estimate  the  workload  for  each  task.  The  workload  com¬ 
ponent  scales  previously  used  in  the  AH-64A  workload  analysis 
(Szabo  &  Bierbaum,  1986)  were  refined  and  used  for  this 
research.  Two  changes  to  the  scales  used  by  Szabo  and 
Bierbaum  were  introduced.  The  7-point  ordinal  scales  used  in 
the  AH - 6 4 A  research  were  converted  to  7-point  interval  scales 
using  the  meuhod  of  pair  comparisons  (Engen,  1971),  and  a 
visual  aided  (night  vision  goggles)  workload  component  was 
added . 

Two  analyses  reached  consensus  in  selecting  the  verbal 
anchor  that  best  matched  the  short  verbal  descriptor  of 
workload  for  each  task  in  the  analysis.  The  consensual 
matches  and  accompanying  ratings  were  subsequently  reviewed 
by  UH-60A  SMEs . 

Estimate  the  time  required  to  perform  each  task.  Each 
task  was  identified  as  discrete  fixed,  discrete  random,  or 
continuous  by  the  researchers  with  the  assistance  of  UH-60A 
SMEs.  The  UH-60A  simulator  was  not  available  for  research 
during  the  time  of  this  project;  therefore,  the  time  assigned 
to  each  task  was  developed  from  SME  estimates  based  on  their 
experiences  in  performing  the  tasks. 
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Phase  2 .  In  developing  the  UH-60A  crewmember  workload 
model,  the  specific  tasks  identified  during  the  Phase  1 
analysis  were  provided  to  the  computer  programmer  to  estab¬ 
lish  the  data  files.  The  researchers  developed  the  decision 
rules  required  to  combine  specific  tasks  to  form  the  unique 
functions.  Decision  rules  were  then  written  to  combine  the 
unique  functions  into  segments.  All  decision  rules  were  then 
programmed  and  the  model  exercised  to  provide  estimates  of 
workload  at  each  half-second  interval  for  each  of  the  34 
segments  in  the  analysis.  A  draft  technical  report  entitled, 
"A  Comprehensive  Task  Analysis  of  the  UH-60A  Mission  With 
Crew  Workload  Estimates  and  Decision  Rules  for  Developing  a 
UH-60  Workload  Prediction  Model,"  has  been  completed  and  is 
being  reviewed. 

Phase .  3 .  A  draft  of  the  Phase  3  work  required  for  the 
CH-47E  mission/task  analysis  has  been  completed.  Presently, 
the  draft  mission/task  analysis  is  being  reviewed  for 
accuracy  by  CH-47D  SMEs. 


Work  Projected 


The  work  required  to  complete  Phase  3  will  be  reviewed 
and  modified  to  incorporate  changes  suggested  as  a  result  of 
the  SME  review.  The  decision  rules  required  for  Phase  4  will 
be  written  and  programmed  to  develop  the  CH-47E  workload 
prediction  model. 


Detailed  descriptions  of  the  proposed  new  MH-60K  and  MH- 
47E  Integrated  Avionics  Subsystem  have  been  received  and  are 
being  reviewed.  Upon  completion  of  the  review,  an  analysis 
of  the  new  tasks  will  be  performed.  The  UH-60A  and  CH-47D 
workload  computer  models  will  then  be  exercised  in  Phases  5 
and  6  to  predict  the  impact  the  design  modifications  are 
likely  to  have  on  the  MH-60K  and  MH-47E  crew  workload. 
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DESIGN  AND  APPLICATION  OF  FLIGHT  SYMBOLOGY 
Dr.  Richard  Weeter,  Project  Director 


I 

j  Background 

j  The  AH-64  attack  helicopter  is  the  first  Army  aircraft 

!  to  feature  flight  and  weapon  systems  designed  to  enable 

;  crewmembers  to  conduct  missions  at  night  and  under  adverse 

weather  conditions.  The  two  systems  that  provide  these 
capabilities  are  the  Pilot  Night  Vision  System  (PNVS)  and  the 
Target  Acquisition  and  Designation  System  (TADS) . 

!  The  PNVS  is  designed  to  provide  the  pilot  with  flight 

•  information  needed  to  operate  the  helicopter  at  night  and 

[  under  degraded  visibility  conditions.  The  primary  PNVS 

f  display  is  a  one-inch  diameter  cathode  ray  tube  (CRT)  mounted 

|  on  the  pilot's  helmet  that  projects  a  collimated  image  to  a 

‘  combiner  lens  in  front  of  the  pilot's  right  eye.  Information 

!  presented  on  the  display  includes  (a)  infrared  imagery  from 

l  the  night  vision  S3nsor  with  a  40°  horizontal  by  30°  vertical 

j  field  of  view,  and  (b)  flight  or  weapon  systems  symbology 

i  overlaid  on  the  imagery. 

* 
i 

I  The  PNVS  symbology  set  consists  of  27  alphanumeric  and 

t  shape-coded  symbols.  Many  of  the  symbols  are  adaptations  of 

!  traditional  electromechanical  instruments  (e.g.,  the  heading 

•  scale)  and  are  located  in  fixed  positions  on  the  displays. 

|  Symbols  representing  aircraft  heading,  airspeed,  altitude, 

r  engine  torque,  and  certain  other  basic  flight  information  are 

constantly  available  on  the  displays.  Other  symbols  repre- 
|  sent  spatial  locations  such  as  the  projected  center  line  of 

the  aircraft  or  the  computed  impact  points  of  weapons.  Such 
\  symbols  may  appear,  disappear,  and  move  on  and  off  the 

displays,  reflecting  changes  in  helicopter  or  sensor 
J  orientation. 

1  To  reduce  obscuration  and  make  the  information  more 

[  task-specific,  four  subsets  or  modes  of  the  PNVS  symbology 

(  can  be  selected  by  the  crewmember.  Each  mode  adds  symbology 

I  to  the  basic  flight  information  that  is  critical  for  per- 

forming  a  flight  or  weapons  task  in  that  mode.  For  example, 
the  hover  mode  adds  a  velocity  vector  and  an  acceleration  cue 
■  to  aid  the  pilot  in  maintaining  a  hover.  Selection  of  the 

•  transition  mode  adds  a  horizon  line  to  the  hover  mode  subset; 

1  the  transition  mode  is  used  when  changing  from  a  hover  to 

;  cruise  flight.  Once  cruise  flight  has  been  established, 

'  selection  of  the  cruise  mode  removes  the  velocity  vector  and 

acceleration  cue,  adding  only  the  horizon  line  to  the  basic 
|  flight  information.  To  aid  the  pilot  in  returning  to  or 


remaining  over  a  chosen  location  with  a  specific  heading,  the 
bob-up  mode  adds  the  velocity  vector,  acceleration  cue, 
command  heading,  and  hover  position  symbols  to  the  basic 
flight  information. 

The  TADS  is  designed  to  provide  the  copilot/gunner  (CPG) 
with  information  needed  for  target  search,  detection,  recog¬ 
nition,  and  designation.  The  primary  TADS  display  is  an 
optical  relay  tube  (ORT)  mounted  on  a  support  with  hand 
controls  located  directly  in  front  of  the  CPG  crew  position. 
The  CPG  may  select  targeting  information  in  the  form  of 
imagery  from  a  forward-looking  infrared  (FLIR)  sensor,  a 
daylight  television  (DTV)  sensor,  or  from  direct  view  optics 
(DVO) .  In  the  FLIR  and  DTV  modes,  flight  and  weapon  symbol¬ 
ogy  is  superimposed  on  the  display  to  provide  additional 
information  required  by  the  CPG. 

The  TADS  symbology  set  consists  of  17  symbols.  Fifteen 
of  these  are  similar  to  symbols  used  in  the  PNVS  set,  but 
they  may  differ  in  size,  location  on  the  display,  or  exact 
meaning.  Two  additional  symbols  are  unique  to  the  TADS  set. 
The  selected  sensor  symbol  is  an  alphanumeric  descriptor  of 
the  sensor  system  (FLIR,  DTV,  or  DVO)  currently  selected  by 
the  CPG.  The  f ield-of-view  gates  identify  the  area  of  the 
display  that  can  be  selected  for  the  next  higher  level  of 
magnification.  In  contrast  to  the  PNVS,  there  are  no  symbol 
subsets  or  modes  for  the  various  functions  performed  by  the 
TADS  . 

During  the  AH-64  development  program,  empirical  research 
comparing  different  symbology  formats  was  found  to  be  "sorely 
lacking"  (Buckler,  1978a).  Furthermore,  Buckler  (1978b) 
noted  that  reconf igurable  simulators  on  which  to  test 
alternative  designs  were  of  limited  availability.  As  a 
result,  the  current  AH-64  symbology  was  designed  largely  on 
the  basis  of  subject  matter  expert  (SME)  opinion.  Buckler 
(1978b)  reported  that  SMEs  conducted  an  analysis  of  the 
information  needed  by  the  crew  to  perform  representative  AH- 
64  flight  and  weapons  tasks.  Following  the  analysis,  the 
SMEs  recommended  the  existing  symbology  format. 

No  evaluation  has  been  made  to  determine  whether  the 
existing  AH-64  symbology  enhances  or  degrades  crew  perfor¬ 
mance  in  critical  helicopter  flight  and  weapons  tasks. 

Viable  design  alternatives  for  helicopter  flight  and  weapons 
symbology  have  not  been  proposed  and  evaluated.  Therefore, 
the  standard  for  the  design  of  symbology  for  future  heli¬ 
copters  is  being  patterned  after  the  existing  AH-64  symbology 
(Department  of  Defense,  1984) . 


Need 


I 

'  A  need  exists  to  develop  a  methodology  to  empirically 

evaluate  display  symbology  and  to  recommend  optimal  symbology 
suites  for  various  Army  helicopter  systems.  Critical  design 
issues  to  be  addressed  during  the  research  include: 

•  the  appropriate  use  of  coding, 

•  the  degree  that  overlaid  symbology  obscures  critical 
visual  information, 

•  the  appropriate  synbol  density,  and 

•  the  compatibility  of  proposed  symbology  with  the 
cognitive  processes  of  the  crewmember. 

Ideally,  the  application  of  the  methodology  will  result  in 
the  design  of  symbology  suites  that  reflect  known  capa¬ 
bilities  and  limitations  of  aviators,  and  provide  relevant 
information  in  formats  that  foster  accurate  and  efficient 
interpretation.  In  addition,  the  resultant  symbology  design 
should  complement  rather  than  degrade  vital  information 
available  from  either  the  natural  external  visual  scene  or 
sensor-provided  imagery. 

The  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  was  tasked  by  the  Army 
Aviation  Systems  Command  (AVSCOM)  to  (a)  develop  the  required 
research  methodology,  and  (b)  apply  the  methodology  in  the 
evaluation  of  existing  helicopter  display  symbology.  Anacapa 
Sciences,  Inc.,  personnel  began  work  on  the  project  in 
February  1987. 


Project  Objectives 

The  three  objectives  of  this  project  are:  (a)  conduct 

an  empirical  evaluation  of  the  existing  AH-64  symbology,  (b) 
identify  potential  deficiencies  and  recommend  improvements  to 
the  AH-64  symbology,  and  (c)  develop  a  methodology  for 
evaluating  alternative  symbology  suites  for  future  Army 
helicopters  . 


Project  Status 


Work  Completed 

Work  on  this  project  commenced  with  a  comprehensive 
review  of  the  aircraft  display  symbology  literature.  Special 
emphasis  was  placed  on  reviewing  research  that  evaluated  the 
effectiveness  of  symbology  coding  dimensions. 


Project  personnel  also  reviewed  available  technical 
documents  and  observed  academic  classes  at  the  U.S.  Army 
Aviation  Center  (USAAVNC) .  Videotapes  were  recorded  from  the 
AH-64  Combat  Mission  Simulator  (CMS)  displays,  and  the 
symbology  and  imagery  available  to  AH-64  crewmembers  during 
representative  flight  and  weapons  tasks  were  reviewed. 
Discrepancies  discovered  in  these  sources  made  it  necessary 
to  compile  detailed  and  accurate  technical  descriptions  of 
the  existing  AH-64  symbology.  Information  from  the  AH-64  CMS 
was  established  as  most  representative  of  the  symbology  set 
used  in  the  actual  aircraft . 

Development  of  a  comprehensive  research  plan  is  underway 
in  which  six  tasks  that  must  be  accomplished  to  meet  the 
project  objectives  have  been  identified.  The  tasks  are 
listed  below: 

•  identify  flight  tasks  that  depend  on  the  symbolic 
information, 

•  develop  flight  scenarios  containing  flight  tasks 
identified  in  the  previous  task, 

•  develop  performance  measures  for  the  flight  tasks 
identified, 

•  identify  equipment  and  other  support  requirements  for 
the  research, 

•  select  and  train  subjects,  and 

•  perform  the  research  needed  to  evaluate  candidate 
symbology  suites. 

A  preliminary  review  has  been  completed  of  the  ARIARDA 
resources  available  to  support  the  research.  The  resources 
include  (a)  a  UH-1  Training  Research  Simulator  (TRS)  modified 
to  provide  simulated  external  visual  scenes,  (b)  independent 
graphics  workstations,  (c)  a  crew  station  procedures  trainer, 
and  (d)  microcomputer-based  part-task  training  devices.  The 
UH-1  TRS  is  expected  to  be  operational  by  the  end  of  calendar 
year  1987  and  has  been  identified  as  the  most  suitable  test¬ 
bed  for  the  project.  To  maximize  the  availability  of  the 
TRS,  it  may  be  possible  to  train  subjects  on  a  microcomputer 
part-task  training  device  presently  being  developed  as  part 
of  a  related  research  project  (see  Ruffner,  1987) . 


Once  the  research  plan  has  been  approved  by  the 
Contracting  Officer's  Technical  Representative  (COTR) ,  work 
will  begin  on  the  tasks  identified  previously.  It  is 


anticipated  that  a  series  of  experiments  will  be  designed  to 
evaluate  candidate  symbology  suites. 


The  first  experiments  will  evaluate  simple  flying  task 
problems,  with  primary  flight  information  presented  through 
subsets  of  the  AH-64  symbology  suite.  Basic  subsets  of 
alternative  symbology  will  also  be  evaluated  using  the  same 
simple  flying  task  problems.  As  the  research  progresses, 
more  complex  experiments  will  be  designed  to  include  multiple 
flying  tasks,  with  more  information  presented  through  symbol¬ 
ogy.  The  experiments  will  be  designed  such  that  performance 
measures  can  be  used  to  compare  the  properties  of  the  AH-64 
symbology  suite  with  alternative  symbology  suites. 

The  successful  accomplishment  of  this  research  should 
result  in  the  development  and  use  of  a  prototype  set  of 
procedures,  equipment,  and  software  to  evaluate  AH-64  and 
alternative  symbology  sets.  The  methods  developed  and 
experience  gained  in  performing  this  research  can  then  be 
employed  to  evaluate  proposed  symbology  for  future  aircraft 
systems . 
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NEW  FAST  DEVEL0PMEN1  AND  VALIDATION 
Dr.  D.  Michael  McAnulty,  Project  Director 


Background 

The  Army's  original  selection  battery,  the  Flight 
Aptitude  Selection  Test  (FAST) ,  was  developed  in  response  to 
the  unacceptably  high  attrition  rates  in  the  flight  training 
program  during  the  1950s.  There  were  actually  two  over¬ 
lapping  FAST  batteries,  one  for  officer  applicants  and  one 
for  enlisted  and  civilian  applicants  to  the  Warrant  Officer 
Candidate  (WOC)  program.  Each  battery  yielded  a  fixed  wing 
and  a  rotary  wing  aptitude  score  for  each  applicant  (Kaplan, 
1965) .  The  FAST,  implemented  in  1966,  resulted  in  a  substan¬ 
tial  reduction  in  the  flight  training  attrition  rates. 

In  1975,  the  U.S.  Army  Aviation  Center  requested  a 
revision  of  the  FAST  due  to  (a)  a  decrease  in  the  validity  of 
the  FAST,  (b)  the  large  number  of  errors  in  scoring  the  FAST, 
(c)  the  excessive  amount  of  time  (about  4  hours)  required  to 
administer  the  FAST,  and  (d)  the  elimination  of  fixed  wing 
training  for  initial  entry  students.  The  goal  of  the  revi¬ 
sion  was  to  develop  a  single,  effective  battery  with  fewer, 
shorter,  and  more  reliably  scored  subtests  (Eastman  & 
McMullen,  1978) . 

The  methodological  approach  chosen  for  the  revision  was 
to  select  the  most  effective  subtests  from  the  FAST  and  then 
to  select  the  most  effective  items  from  each  subtest  for 
inclusion  in  a  revised  FAST  (RFAST) .  Factor  analyses  and 
multiple  regression  analyses  were  used  to  select  7  of  the  12 
FAST  subtests  for  retention.  Subsequently,  item  difficulties 
and  item  discrimination  coefficients  were  analyzed  to  select 
the  specific  items  to  be  retained  in  each  subtest.  The 
RFAST,  which  is  approximately  one-half  the  length  of  the 
FAST,  was  implemented  in  1980. 

Subsequently,  research  was  conducted  to  evaluate  the 
reliability  and  validity  of  the  RFAST  and  its  subtests. 
Lockwood  and  Shipley  (1984)  found  that  six  of  the  seven 
subtests  had  adequate  internal  consistency  and  that  the 
correlation  between  the  RFAST  score  and  performance  in 
initial  entry  rotary  wing  (IERW)  training  was  statistically 
significant.  They  concluded,  however,  that  the  low  percent¬ 
age  of  variance  accounted  for  by  the  RFAST  indicates  the 
battery  has  limited  utility  in  predicting  IERW  performance. 

In  addition,  Smith  and  McAnulty  (1985)  found  that  the  RFAST 
has  marginal  retest  reliability  and  that  there  was  a  large 


increase  in  the  average  score  on  retesting,  indicating  a  need 
for  an  equivalent  form  for  use  when  retesting  is  required. 


The  first  phase  of  research  in  the  development  of  a  new 
FAST  (NFAST)  was  to  identify  the  ability  requirements  for 
successful  completion  of  IERW.  Experienced  IERW  instructor 
pilots  (IPs)  were  asked  (a)  to  identify  the  tasks  that  are 
most  indicative  of  successful  performance  in  the  primary  and 
instrument  phases  of  IERW,  and  (b)  to  judge  the  type  and 
importance  of  the  abilities  that  are  required  to  perform  each 
task.  The  task-ability  ratings  for  each  IP  were  then  trans¬ 
formed  to  a  normally  distributed,  equal-interval  scale  using 
the  method  of  successive  intervals  (McAnulty  &  Jones,  1984)  . 
Analyses  of  the  transformed  ratings  indicated  that  24  abili¬ 
ties  from  the  psychomotor,  perceptual,  language,  and  cogni¬ 
tive  domains  were  required  for  successful  performance  in 
IERW.  A  test  specifications  matrix  was  then  developed  as  a 
guide  in  developing  the  NFAST  (McAnulty,  Jones,  Cohen,  & 
Lockwood,  1984) . 

In  the  last  phase  of  the  previous  research,  a  battery  of 
nine  tests  was  developed  to  measure  11  of  the  required  cogni¬ 
tive  and  perceptual  abilities.  The  seven-hour  experimental 
battery  was  administered  to  290  general  population  subjects 
at  three  military  installations  in  the  southeastern  United 
States  (McAnulty,  1986)  . 


Need 

The  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  has  a  continuing  requirement 
to  evaluate  and  improve  the  tests  that  are  used  to  select 
applicants  for  the  Army  IERW  training  program.  As  indicated 
in  the  background  section,  a  new  FAST  battery  is  needed  to 
improve  the  reliability  and  validity  of  the  IERW  selection 
process  and  to  develop  an  alternate  form  to  retest  IERW 
applicants . 


Project  Objectives 


The  general  objective  of  this  project  is  to  develop, 
evaluate,  and  implement  a  more  effective  battery  of  IERW 
selection  tests.  The  specific  technical  objectives  of  the 
research  project  are  to: 


•  evaluate  the  performance  of  general  population 

subjects  on  an  experimental  battery  of  tests  designed 
to  measure  the  abilities  that  are  required  for 
successful  performance  in  IERW, 


•  develop  two  alternate  forms  of  the  NFAST  battery, 

•  conduct  research  to  validate  and  equate  the  alternate 
forms  of  the  NFAST  battery, 

•  revise,  produce,  and  implement  the  operational 
versions  of  the  NFAST  battery,  and 

•  evaluate  the  NFAST  battery  and  administrative 
procedures  in  operational  use. 


This  project  is  a  continuation  of  the  ongoing  ARIARDA 
research  program  in  aviator  selection  and  classification 
(McAnult.v,  1986)  .  The  current  research  is  being  conducted  in 
three  phases.  In  Phase  1,  the  data  collected  during  the 
experimental  administration  of  the  new  battery  will  be 
analyzed  to  determine  the  psychometric  characteristics  of  the 
nine  tests  and  the  interrelationships  among  the  tests.  The 
results  of  these  analyses  will  be  used  to  develop  two  equiva¬ 
lent  forms  of  an  NFAST  validation  battery. 

Phase  2  of  the  research  project  is  a  traditional  predic¬ 
tive  validity  study.  The  purposes  of  the  validation  study  are 

(a)  to  determine  the  relationship  between  the  NFAST  tests, 
other  predictor  data,  and  performance  in  IERW  training,  and 

(b)  to  equate  the  alternate  forms  of  the  battery  on  a  large 
sample  drawn  from  the  target  population  of  flight  students. 
Phase  2  of  the  research  also  requires  the  development  and 
evaluation  of  measures  of  IERW  performance.  The  results  of 
the  validation  analyses  will  be  used  to  produce  two  alternate 
forms  of  an  operational  NFAST  battery. 

Phase  3  of  the  research  project  will  be  the  imple¬ 
mentation  of  the  NFAST.  All  ancillary  materials  (machine 
scorable  answer  sheets,  administrative  manuals,  scoring  and 
equating  manuals,  selection  criteria)  will  be  developed  and 
delivered  for  operational  use.  After  the  NFAST  is  imple¬ 
mented,  a  sample  of  field  data  will  be  analyzed  to  ensure 
that  the  psychometric  characteristics  of  the  operational 
battery  are  not  significantly  different  from  the  validation 
battery.  Finally,  data  will  be  collected  and  analyzed  to 
ensure  that  the  tests  are  being  properly  administered  and  the 
test  scores  are  being  used  appropriately  in  the  selection 
process  . 


The  statistical  analyses  of  the  experimental  NFAST 
battery  have  been  completed.  The  experimental  battery 
consisted  of  eight  tests  that  were  each  designed  to  measure  a 
unique  ability  and  one  test  that  was  designed  to  measure  a 
complex  of  abilities  required  for  the  successful  completion 
of  IERW  training.  The  results  indicate  that  the  complex 
ability  test  and  six  of  the  unique  ability  tests  assess 
reliable  individual  differences  in  the  abilities  of  interest. 
The  average  difficulty  levels  of  the  seven  tests  are  near  the 
optimum  level  of  .50;  the  test  variances  indicate  the  mea¬ 
surement  of  substantial  individual  differences;  and  the 
estimates  of  reliability  are  acceptabDe  when  test  length  and 
the  design  specifications  are  considered.  The  remaining  two 
unique  ability  tests  had  undesirable  psychometric  character¬ 
istics  or  dia  no*  contribute  any  unique  variance  to  the 
factor  structure  of  the  battery.  A  technical  report 
(McAnulty,  Cross,-  &  Jones,  1986)  that  describes  the  results 
of  this  research  was  written  and  submitted  to  ARIARDA  for 
review . 

The  results  of  the  Phase  1  research  were  used  to  develop 
two  alternate  forms  of  the  NFAST  validation  battery.  Each 
form  consists  of  modified  versions  of  the  complex  test  and 
the  six  unique  ability  tests  with  acceptable  characteristics. 
In  general,  the  validation  battery  tests  are  approximately 
two-thirds  the  length  of  the  experimental  battery  tests.  The 
alternate  forms  of  five  of  the  tests  have  approximately  50% 
of  the  items  in  common.  The  complex  ability  test  forms  and 
one  of  the  unique  ability  test  forms  do  not  have  any  identi¬ 
cal  items.  Finally,  a  knowledge  test  of  helicopter  opera¬ 
tions  and  aerodynamic  principles  was  adapted  from  the  RFAST 
battery  for  inclusion  in  the  validation  battery.  The  items 
on  the  knowledge  test  are  identical  on  both  forms.  Each  form 
of  the  validation  battery  requires  approximately  four  hours 
to  administer. 


The  Phase  2  research  activities  are  partially  completed. 
Between  March  and  September  1987,  the  NFAST  battery  was 
administered  to  approximately  95%  of  the  entering  commis¬ 
sioned  officer  (CO)  and  WOC  f] ight  students  within  the  first 
two  weeks  of  IERW  training.  When  the  test  administration 
phase  is  terminated,  complete  test  data  will  be  collected 
from  approximately  350  CO  and  350  WOC  students . 


Preliminary  test  data  analyses  indicate  that  target 
population  performance  on  the  validation  battery  (excluding 
the  helicopter  knowledge  test)  is  similar  to  the  general 
population  performance  on  the  experimental  battery:  the 
average  difficulty  levels  are  near  .50  despite  the  more 
restrictive  time  limits  that  were  imposed  on  the  validation 
tests,  and  the  variances  indicate  that  substantial  individual 
differences  in  ability  are  being  measured  by  the  tests. 
Performance  on  the  two  forms  of  the  battery  are  very  similar 
except  for  one  of  the  unique  ability  tests.  Test  performance 
by  the  CO  and  WOC  students  is  also  quite  similar,  although 
there  are  statistically  significant  differences  between  them: 
CO  students  score  significantly  higher  on  three  of  the  tests 
and  WOC  students  score  significantly  higher  on  one  of  the 
tests . 

The  preliminary  test  results  on  the  helicopter  knowledge 
test  adapted  from  the  RFAST  indicate  that  the  test  is  not 
difficult  (the  average  difficulty  level  is  .81)  and  there  is 
limited  variability  in  the  scores.  WOC  students  score 
significantly  higher  on  the  test  than  the  CO  students.  How¬ 
ever,  there  is  no  difference  in  performance  by  either  student 
group  on  the  two  forms  of  the  test  .  Since  the  test  items  are 
identical  on  both  forms,  this  result  indicates  there  was  no 
systematic  sampling  bias  in  terms  of  aviation-related 
knowledge  in  the  assignment  of  students  to  the  alternate 
forms  of  the  NFAST  battery. 

The  collection  of  IERW  performance  measures  has  also 
started.  Student  grades  on  academic  tests  and  end-of-IERW- 
phase  flight  checkrides  are  routinely  reviewed  as  they  are 
collected  from  the  Aviation  Management  Information  System. 

In  addition,  changes  to  the  class  rosters  are  monitored  to 
identify  tested  students  who  have  been  eliminated  from 
training  or  set  back  to  a  different  class.  The  reasons  for 
these  administrative  changes  (e.g.,  some  setbacks  may  be  due 
to  student  flight  deficiencies  and  others  may  be  due  to  a 
shortage  of  IPs)  are  also  obtained  from  the  class  rosters. 
Finally,  efforts  have  been  initiated  to  arrange  for  diag¬ 
nostic  interviews  with  tested  students  who  have  been  elimi¬ 
nated  from  IERW  training.  The  purpose  of  the  interview  is  to 
explore,  in-depth,  the  problems  the  student  had  during  flight 
training  so  that  specific  deficiencies  can  be  identified  and 
related  to  individual  test  performance. 


The  NFAST  battery  will  continue  to  be  administered  to 
incoming  1ERW  students  until  the  minimum  number  of  complete 
test  data  sets  is  collected  for  each  group  and  for  each  form. 
Past  experience  indicates  that  the  data  from  approximately 
10%  of  the  subjects  will  be  eliminated  from  the  primary 
analyses  because  of  mitigating  subject  factors  (e.g.,  ill¬ 
ness,  lack  of  sleep  due  to  standing  duty  watch,  failure  to 
follow  directions  or  provide  complete  data,  or  observed  or 
self-reported  lack  of  effort) .  When  the  test  data  base  has 
been  established,  statistical  analyses  will  be  conducted  to 
determine  the  psychometric  characteristics  of  the  tests  and 
test  items,  the  equivalence  of  the  alternate  test  forms,  and 
the  interrelationships  among  the  tests  and  other  predictor 
data  (e.g.,  biographical  information  and  RFAST  scores) . 

IERW  performance  data  will  be  collected  until  all  the 
students  in  the  test  data  base  have  either  graduated  or 
attrited  from  flight  training  (approximately  11  months  after 
the  last  battery  administration) .  The  performance  data  will 
include  academic  grades,  checkride  scores,  flight  hours, 
administrative  changes  (e.g.,  setbacks,  additional  flight 
hour  authorizations,  mission  track  assignment) ,  and  terminal 
status  (pass  or  fail  and  reason  for  failure) .  In  addition, 
information  obtained  from  the  diagnostic  interviews  will  be 
incorporated  into  the  performance  data  base .  When  the 
performance  data  base  has  been  established,  appropriate 
statistical  analyses  will  be  conducted  to  determine  the 
psychometric  characteristics  of  the  performance  measures,  the 
interrelationships  among  the  performance  measures,  and  the 
correlations  between  the  predictor  and  performance  measures. 

The  results  of  these  analyses  will  be  used  to  produce 
two  alternate  forms  of  the  NFAST  for  operational  use.  The 
development  of  the  operational  batteries  should  be  completed 
by  March  1989. 


Phase  3  activities  have  not  been  formally  initiated, 
although  a  prototype  administration  manual  has  been  developed 
for  use  with  the  validation  battery  and  has  been  informally 
evaluated  during  the  Phase  2  administrations.  The  adminis¬ 
tration  manual  will  be  modified  for  use  with  the  operational 
battery.  Additional  ancillary  materials,  such  as  a  scoring 
and  equating  manual  and  selection  criteria  guidelines,  will 
be  developed.  The  operational  batteries  and  ancillary 


materials  will  be  delivered  to  the  U.S.  Army  Soldier  Support 
Center  for  implementation. 


After  the  NFAST  is  in  operational  use,  follow-on 
research  is  planned  to  ensure  that  applicant  performance  on 
the  batteries  is  within  acceptable  limits,  that  administra¬ 
tive  procedures  are  being  followed,  and  that  the  selection 
criteria  are  valid.  Depending  on  the  results  of  the 
.  preceding  research,  it  may  be  necessary  to  conduct  a  second 

validation  study  using  an  unrestricted  sample  (i.e.,  not 
already  selected  for  flight  training)  of  IERW  applicants. 
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THE  ENLISTED  AVIATOR  PROGRAM  SURVEY 


Dr.  D.  Michael  McAnulty  and  Dr.  John  W.  Ruffner, 
Project  Directors 


In  February  1986,  Anacapa  Sciences,  Inc.,  was  tasked  by 
the  Army  Research  Institute  Aviation  Research  and  Development 
Activity  (ARIARDA)  to  provide  research  support  to  a  Joint 
Working  Group  (JWG)  at  the  U.S.  Army  Aviation  Center 
(USAAVNC) ,  Fort  Rucker,  Alabama.  The  JWG  was  established  to 
meet  a  Department  of  the  Army  (DA)  requirement  to  determine 
the  feasibility  and  desirability  of  utilizing  enlisted 
personnel  as  Army  aviators. 

The  possibility  of  establishing  an  enlisted  aviator  (EA) 
career  field  was  examined  previously  and  rejected  as  not 
being  a  viable  concept  (DA  Aviation  Special  Task  Force, 

1977) .  At  the  time  the  current  project  was  initiated,  how¬ 
ever,  the  Army  warrant  officer  force  structure  requirements 
for  1992  exceeded  the  budgeted  end  strength  by  more  than 
3,000  personnel.  If  feasible  and  desirable,  the  utilization 
of  enlisted  personnel  as  Army  aviators  would  enable  the  Army 
to  meet  the  budgeted  end  strength  restrictions  and  to 
maintain  adequate  staffing  of  the  aviation  units. 

Concern  about  the  shortage  of  warrant  officers  in  the 
1990s  and  several  apparent  shortcomings  in  the  previous  EA 
research  prompted  the  DA  to  direct  that  another  comprehensive 
study  be  conducted  to  evaluate  all  the  issues  related  to  the 
implementation  of  an  EA  military  occupational  speciality 
(MOS) .  The  JWG  developed  a  multifaceted  research  approach  to 
study  the  EA  issues.  E’irst,  information  was  collected  from 
the  U.S.  Navy  and  Air  Force,  and  from  allied  countries  on 
their  experience  with  enlisted  aviators.  Second,  operations 
research  analysts  conducted  a  cost-benefit  analysis  of  an  EA 
MOS  under  varying  assumptions  of  paygrade,  flight  pay, 
training  attrition  and  remediation,  and  personnel  retention 
rates.  Third,  members  of  the  JWG  examined  the  legal,  regula¬ 
tory,  and  conventional  ramifications  of  utilizing  EAs . 

Fourth,  comparative  information  (e.g.,  on  officers  and 
enlisted  personnel  performing  similar  jobs  in  other  career 
fields)  was  collected  to  make  inferences  about  the  potential 
for  operational  effectiveness  and  to  evaluate  the  risks  to 
safety  that  would  result  from  utilizing  EAs. 

Finally,  the  JWG  tasked  ARIARDA  and  Anacapa  with  the 
development,  administration,  and  analysis  of  a  series  of 


surveys  designed  to  obtain  broad-based  subjective  opinions 
about  the  feasibility  and  desirability  of  utilizing  EAs  in 
Army  aviation.  As  guidance  to  the  Anacapa  research  team,  the 
JWG  provided  a  matrix  of  respondent  samples  and  management 
issues  to  be  addressed  by  the  surveys.  The  JWG  also  provided 
descriptions  of  three  options  for  evaluation  as  an  EA 
program.  Under  Option  1  (total  conversion),  the  warrant 
officer  corps  would  be  completely  replaced  over  a  period  of 
years  by  an  EA  MOS .  Under  Option  2  (partial  conversion) ,  new 
aviators  would  serve  in  enlisted  paygrades  for  six  years  and 
then  be  eligible  for  promotion  to  warrant  officer.  Under 
Option  3  (Air  Weapons  Operator) ,  EAs  would  augment  the 
warrant  officer  corps  by  performing  a  limited  range  of 
aviation  duties. 


Project  Objectives 

The  objectives  of  this  project  are  to  collect  subjective 
opinions  from  various  subject  matter  experts  (SMEs)  and 
potential  EA  participants  to: 

•  evaluate  the  feasibility  and  desirability  of  an  EA 
program, 

•  evaluate  the  advantages  and  disadvantages  of  each  of 
the  three  EA  options, 

•  develop  the  policies  that  should  be  adopted  to  manage 
an  EA  program, 

•  evaluate  the  interpersonal  aspects  that  would  affect 
an  EA  program, 

•  evaluate  the  impact  of  an  EA  program  on  aviation 
training,  doctrine,  and  mission  effectiveness,  and 

•  evaluate  the  impact  of  an  EA  program  on  the  accession 
and  retention  of  aviators. 


Research  Approach 

As  directed  by  the  JWG,  all  the  opinion  data  were  to  be 
collected  by  surveying  aviation  SMEs  and  potential  EAs  about 
several  issues  related  to  the  implementation  of  an  EA 
program.  Each  survey  was  developed  in  a  paper-and-penci 1 
response  format.  The  general  approach  to  the  development  of 
each  survey  was  to: 

•  identify  the  target  population  and  develop  a  sampling 
plan, 

•  identify  the  issues  to  be  addressed  by  the  sample, 

•  draft  items  to  collect  the  required  data, 
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•  pretest  the  items  on  a  representative  sample  of 
respondents, 

•  revise  the  items  and  organize  the  survey  form, 

•  obtain  the  approval  of  the  Soldier  Support  Center  to 
use  the  survey,  and 

•  reproduce  the  survey  for  administration. 

The  surveys  were  constructed  using  a  variety  of  item 
formats  and  response  scaling  techniques.  To  the  extent 
possible,  however,  the  items  were  constructed  in  a  multiple 
choice  or  restricted  f ill-in-the-blank  format  that  was 
computer  coded  to  facilitate  data  processing.  After  each 
survey  or  set  of  surveys  were  developed,  they  were  adminis¬ 
tered  to  the  designated  sample.  Data  entry  and  verification 
programs  were  developed  and  used  to  build  a  project  data 
base.  Finally,  the  data  were  analyzed  and  the  results  were 
submitted  to  the  JWG  for  incorporation  into  their  final 
report . 


Survey  Instruments 

Eleven  survey  instruments  were  developed  to  collect  the 
data  required  to  meet  the  project  objectives.  Two  of  the 
surveys  were  independent  and  administered  to  all  members  of 
the  respective  samples.  The  remaining  surveys  were  divided 
into  three  series  of  three  surveys  each;  each  survey  in  a 
series  presented  information  about  only  one  of  the  EA  options 
and  was  administered  to  approximately  one-third  of  the 
members  in  the  respective  series'  sample.  The  survey  instru¬ 
ments  are  described  in  the  following  sections. 


Survey  Form  G-l 

The  primary  research  instrument  (Form  G-l)  was  a  general 
survey  of  opinions  about  the  feasibility  and  desirability  of 
an  EA  program  including  all  three  of  the  EA  options.  Form  G- 
1  was  designed  for  administration  to  senior  noncommissioned 
officers  (NCOs),  aviation  warrant  and  commissioned  officers, 
nonaviation  warrant  and  commissioned  officers,  and  National 
Guard  state  aviation  officers.  Form  G-l  is  divided  into  the 
following  six  parts. 

•  Part  I  contains  30  items  requesting  background  infor¬ 
mation  about  the  respondent's  personal  characteristics 
and  military  experience. 

•  Part  II  contains  44  items  requesting  opinions  about  EA 
policies  and  duties.  The  section  on  policy  covers  the 


selection  and  training  of  EAs,  the  assignment  and 
management  of  EAs,  and  variables  to  be  considered  in 
the  promotion  of  EAs.  The  duties  section  covers 
opinions  on  the  aviation  and  military  responsibilities 
of  EAs,  the  requisite  conditions  for  assigning  EAs  to 
various  aviation  positions,  and  the  types  or  decisions 
EAs  would  be  authorized  to  make. 

•  Part  III  contains  50  items  addressing  the  social 
aspects  of  an  EA  program.  Part  III  is  divided  into 
six  sections  on  authority,  social  status,  social 
issues,  morale,  command  relationships,  and  discipline. 

•  Part  IV  contains  20  items  requesting  opinions  about 
the  impact  that  an  EA  program  would  have  on  aviation 
unit  training,  doctrine,  and  mission  effectiveness. 

•  Part  V  contains  9  items  requiring  21  responses  to 
questions  about  the  retention  of  EAs  beyond  the  period 
of  initial  obligated  service  under  each  of  the  three 
options  that  were  identified  previously. 

•  Part  VI  contains  9  items  requesting  information  about 
the  respondent's  summary  opinions  of  each  EA  option. 

In  addition,  a  24-item  supplement  to  Form  G-l  was 
prepared  for  use  with  the  National  Guard  state  aviation 
officers.  The  supplement  was  designed  to  collect  information 
on  the  training,  recruitment,  and  retention  of  aviators  that 
was  specific  to  the  National  Guard. 


Survey  Form  RQ-1 

Survey  Form  RQ-1  was  designed  to  collect  information 
from  Army  recruiters  about  the  potential  for  attracting 
qualified  personnel  for  each  of  the  EA  options.  The 
recruiter  survey  was  originally  designed  to  be  in  a  telephone 
interview  format,  but  was  subsequently  changed  to  a  paper- 
and-pencil  response  format.  Form  RQ-1  is  divided  into  five 
parts.  Part  I  contains  7  items  requesting  general  informa¬ 
tion  about  the  individuals  that  the  respondent  currently 
counsels  during  a  typical  12  month  period.  Parts  II  through 
IV  contain  8  items  requesting  information  on  the  recruitment 
of  personnel  under  options  1,  2,  and  3,  respectively.  Part  V 
contains  5  items  requesting  demographic  information  about  the 
respondent  . 

Series  WO 


Series  WO  comprises  three  surveys  that  were  designed  to 
collect  opinion  information  from  warrant  officers  and  warrant 
officer  candidates  about  the  feasibility  and  desirability  of 


an  EA  program.  Each  survey  (OP-1,  OP-2,  and  OP-3)  in  the 
series  presents  a  description  of  only  one  of  the  three  EA 
options.  The  respondents  are  asked  to  provide  demographic 
information  (18  items)  about  themselves,  to  study  the 
description  of  the  EA  option,  and  to  answer  12  questions 
about  their  opinions  of  the  described  option. 


Series  EP  comprises  three  surveys  that  were  designed  to 
collect  opinion  information  from  enlisted  personnel  who  are 
in  the  potential  flight  school  applicant  pool  about  the 
feasibility  and  desirability  of  an  EA  program.  Each  survey 
(OP-1 ,  OP-2,  and  OP-3)  in  the  series  presents  a  description 
of  only  one  of  the  three  EA  options.  The  respondents  are 
asked  to  provide  demographic  information  (26  items)  about 
themselves,  to  study  the  description  of  the  EA  option,  and  to 
answer  26  questions  about  their  opinions  of  the  described 
option . 


Series  HS  comprises  three  surveys  that  were  designed  to 
collect  opinion  information  from  high  school  students  who  are 
in  the  potential  flight  school  applicant  pool  about  the 
feasibility  and  desirability  of  an  EA  program.  Each  survey 
(OP-1,  OP-2,  and  OP-3)  in  the  series  presents  a  description 
of  only  one  of  the  three  EA  options.  The  respondents  are 
asked  to  provide  demographic  information  (21  items)  about 
themselves,  to  study  the  description  of  the  EA  option,  and  to 
answer  23  questions  about  their  opinions  of  the  described 
option . 


Form  G-l 

The  general  opinion  survey  (Form  G-l)  was  administered 
to  297  senior  NCOs,  193  nonaviation  officers  (163  commis¬ 
sioned  and  30  warrant),  and  291  aviation  officers  (169  active 
duty  and  46  National  Guard  commissioned,  and  76  warrant). 

The  NCOs  were  in  paygrades  E6  through  E9,  the  warrant 
officers  were  in  paygrades  W01  through  CW4,  and  the  commis¬ 
sioned  officers  were  in  paygrades  02  through  06. 


After  Form  RQ-1  was  developed,  the  U.S.  Army  Recruiting 
Command  declined  to  participate  in  the  research.  Instead, 
they  agreed  to  submit  a  letter  stating  that  they  would  have 
no  difficulty  in  recruiting  sufficient  numbers  of  qualified 
personnel  to  meet  the  requirements  of  any  of  the  three  EA 
options.  No  further  research  was  conducted  using  Form  RQ-1. 


Series  WO  was  administered  to  180  warrant  officer 
candidates  enrolled  in  the  Initial  Entry  Rotary  Wing  training 
course  and  to  69  aviation  warrant  officers  attending  the 
Warrant  Officer  Advanced  Course.  However,  only  the  WO  data 
collected  from  the  warrant  officer  candidates  were  retained 
for  analysis.  The  bias  resulting  from  the  warrant  officers' 
vested  interest  in  a  continuing  warrant  officer  force 
resulted  in  a  uniform  and  extremely  negative  opinion  toward 
any  EA  program.  The  69  aviation  warrant  officers  expressed 
their  general  opinions  about  an  EA  program  by  completing  Form 
G-l . 


Series  EP  was  administered  to  619  enlisted  personnel  in 
paygrades  El  through  E5.  Most  of  the  respondents  had  served 
less  than  one  enlistment  period.  There  were  251  newly 
inducted  recruits  included  in  the  EP  sample  who  had  served 
less  than  one  week  in  the  Army.  All  the  data  were  utilized 
in  the  analyses. 


Series  HS  was  administered  to  147  high  school  studei^s 
in  the  Fort  Rucker,  Alabama,  area.  However,  an  evaluation  of 
the  demographic  characteristics  of  the  sample  indicated  that 
a  majority  of  the  respondents  did  not  meet  the  sampling 
criteria.  Specifically,  many  of  the  students  were  not 
graduating  seniors,  had  no  interest  in  learning  to  fly,  or 
were  ineligible  for  an  EA  program  because  of  physical  or 
academic  deficiencies.  Because  of  these  problems  and  the 
inclusion  of  a  large  number  of  new  recruits  (many  of  whom 
were  recent  high  school  graduates)  in  the  EP  sample,  the  HS 
data  were  not  analyzed  further. 


The  results  and  conclusions  of  the  EA  surveys  are  based 
on  the  data  from  the  1580  respondents  to  Form  G-l,  Series  WO, 
and  Series  FP .  There  were  substantial  differences  in  the 
opinions  held  by  the  various  groups  that  were  sampled.  For 
example,  the  senior  NCOs  held  more  positive  attitudes  than 
officers  about  the  capabilities  of  enlisted  personnel  per¬ 
forming  as  aviators.  Similarly,  aviation  officers  held  more 
negative  attitudes  than  nonaviation  officers  about  the 
mission  effectiveness  and  training  requirements  for  EAs .  On 
many  issues,  however,  the  respondent  samples  agree  on  the 
direction  (e.g.,  positive  or  negative)  of  their  collective 
opinions,  although  they  may  differ  in  degree  (e.g.,  moderate 
or  extreme  in  opinion) .  Therefore,  there  are  several  general 
conclusions  that  can  be  drawn  from  the  combined  data. 

First,  the  concept  of  an  EA  program  was  judged  to  be 
moderately  feasible  but  only  marginally  desirable.  This 
result  was  interpreted  to  mean  that  an  EA  program  could  be 
effectively  managed  if  required,  but  there  would  be  detri¬ 
mental  consequences  associated  with  the  program.  For 
example,  the  respondents  estimated  that  the  ability  to  meet 
mission  requirements  would  not  change  if  enlisted  personnel 
were  assigned  as  aviators,  but  the  amount  of  time  and 
resources  required  for  training  and  the  amount  of  supervision 
required  in  aviation  units  would  increase.  The  respondents 
also  estimated  that  the  clarity  of  the  command  structure 
would  be  reduced  if  an  EA  program  were  implemented  and  that 
EAs  would  not  have  the  necessary  authority  to  meet  their 
aviation  responsibilities. 

Second,  approximately  75%  of  the  respondents  indicated 
that,  compared  to  other  aviators,  EAs  should  have  (a)  the 
same  entrance  requirements  for  initial  and  advanced  flight 
training,  (b)  the  same  performance  standards  in  training,  (c) 
the  same  training  requirements  for  maintaining  flight 
proficiency,  and  (d)  the  same  standards  for  being  selected  as 
pilot-in-command.  Essentially,  the  respondents  indicated 
that  the  proficiency  of  aviation  personnel  should  not  be 
reduced  because  of  the  lower  paygrades  associated  with  an  EA 
program . 

Third,  the  respondents  to  all  the  surveys  agreed  that 
partial  conversion  was  the  most  feasible  and  desirable  EA 
option.  The  total  conversion  and  Air  Weapons  Operator 
options  were  perceived  as  equally  desirable  by  potential 
applicants.  The  EP  and  WO  respondents  rated  the  desire  to 
fly  and  the  excitement  of  the  job  as  the  primary  reasons  for 
applying  to  flight  school.  However,  the  G-l  respondents 


judged  the  total  conversion  option  as  only  marginally 
feasible  and  not  at  all  desirable.  This  result  probably 
reflects  the  long-term  management  implications  for  a  career 
program  that  has  increasing  levels  of  responsibility  but  only 
limited  promotion  potential.  There  was  no  generalized 
agreement  on  the  feasibility  and  desirability  of  the  Air 
Weapons  Operator  option  by  the  G-l  respondents. 

Finally,  two  conclusions  can  be  drawn  about  EA  recruit¬ 
ment  and  retention.  The  data  support  the  opinion  of  the  Army 
Recruiting  Command  that  sufficient  numbers  of  qualified 
personnel  could  be  recruited  for  an  EA  MOS .  A  majority  of 
the  EP  and  WO  respondents  indicated  that  they  would  very 
likely  apply  for  any  one  of  the  EA  options  if  it  was  the  only 
flight  program  currently  available.  Conversely,  the  G-l  data 
indicated  that  EAs  would  be  more  likely  than  current  aviators 
to  leave  the  Army  after  their  initial  obligated  service.  Pay 
and  benefits,  which  were  the  primary  reasons  cited  by 
enlisted  personnel  for  remaining  in  the  Army,  would  be  lower 
in  each  of  the  three  EA  options. 


As  indicated  above,  the  survey  instruments  have  been 
developed  and  administered,  and  the  data  have  been  analyzed. 
The  results,  including  the  individual  item  analyses  and 
summary  tables  and  figures,  were  submitted  to  the  JWG  for 
incorporation  into  their  briefings  and  final  report.  The 
Anacapa  researchers  also  provided  consulting  services  to  the 
JWG  as  the  study  was  presented  to  various  levels  in  the  chain 
of  command.  The  DA  accepted  the  conclusion  of  the  JWG  that, 
although  the  EA  program  was  feasible,  the  current  aviation 
warrant  officer  force  structure  should  be  retained  for  the 
present  time  and  that  other  alternatives  should  be  pursued  to 
meet  the  budgeted  end  strength  restrictions.  No  further  work 
by  Anacapa  is  projected  on  the  Enlisted  Aviator  Program. 


DEVELOPMENT  OF  A  PEER  COMPARISON  PROGRAM 
Dr.  D.  Michael  McAnulty,  Project  Director 


Background /Need 

This  project  was  initiated  in  response  to  a  request  from 
the  School  Secretary,  U.S.  Army  Aviation  Center,  for  support 
in  developing  an  algorithm  to  select  course  honor  graduates 
based  on  the  "whole  person"  concept .  The  School  Secretary 
wanted  to  augment  the  academic  grade  criterion  used  to  select 
honor  graduates  in  the  Aviation  Officer  Advanced  Course 
(AVNOAC) ,  a  five-month  officer  training  course  for  captains 
and  promotable  first  lieutenants.  The  purposes  of  the  aug¬ 
mented  program  are: 

•  to  motivate  students  to  maximize  their  military  as 
well  as  their  academic  efforts  during  the  course,  and 

•  to  identify  students  who  have  high  potential  as  Army 
aviation  officers  at  an  early  stage  of  their  careers. 

Specifically,  t^®  School  Secretary  was  interested  in 
using  peer  assessments  by  the  AVNOAC  students  as  a  component 
in  the  honor  graduate  selection  algorithm.  The  peer  assess¬ 
ments  were  to  evaluate  aspects  of  the  students’  performance 
that  were  not  reflected  in  their  academic  scores.  Instructor 
ratings  were  not  considered  because  of  the  limited  inter¬ 
action  between  the  school  cadre  and  the  students. 


Project  Objectives 

Following  a  review  of  the  peer  assessment  literature  and 
the  AVNOAC  syllabus,  a  peer  comparison  (PC)  methodology  was 
proposed  for  use  in  the  AVNOAC.  The  School  Secretary  agreed 
to  support  the  following  research  objectives: 

•  identify  the  most  important  military  qualities  that 
could  be  assessed  by  peers  during  the  AVNOAC, 

•  develop  the  PC  instruments  and  procedures  for  use  in 
the  AVNOAC,  and 

•  experimentally  evaluate  the  PC  technique  prior  to 
implementation . 
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Research  Approach 


The  research  approach  was  divided  into  three  phases  that 
correspond  to  the  research  objectives.  Phase  1  involves  the 
administration  of  a  military  qualities  survey  to  identify  the 
most  important  qualities  that  could  be  assessed  by  peers  during 
the  AVNOAC .  The  survey  asks  senior  aviation  officers  to  rate  a 
list  of  primary  military  qualities  as  dimensions  for  evaluating 
student  performance  and  for  identifying  students  with  high 
career  potential.  The  survey  data  provides  the  information 
needed  to  develop  the  PC  instruments. 

Phase  2  involves  the  development  of  three  project 
assessment  instruments:  the  PC  form  to  be  completed  by  the 
class  members  to  evaluate  their  peers,  a  faculty  advisor 
rating  form  to  be  completed  by  each  class  member's  training 
officer,  and  a  student  critique  to  be  completed  by  the 
students  to  evaluate  the  PC  instruments  and  procedures. 

Phase  3  involves  the  experimental  administration  and  evalua¬ 
tion  of  the  PC  technique  in  the  AVNOAC. 
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Phase  1:  Military  qualities  survey.  following  a  search 
of  the  literature  and  a  review  of  current  Army  student  eval¬ 
uation  dimensions,  definitions  of  14  primary  military 
qualities  (e.g.,  adaptability,  initiative,  judgment,  leader¬ 
ship,  and  responsibility)  were  compiled  for  evaluation  by 
senior  aviation  officers.  Several  important  military 
qualities  were  excluded  because  they  are  evaluated  by  aca¬ 
demic  scores  or  are  unlikely  to  be  demonstrated  during  the 
AVNOAC.  The  military  qualities  survey  was  sent  to  16  senior 
Army  officers  who  were  asked  to  rate  each  quality  on  the 
following  four  scales: 

•  importance  to  the  performance  of  captains, 

•  importance  to  the  performance  of  senior  officers, 

•  probability  of  demonstration  during  the  AVNOAC,  and 

•  degree  of  overlap  with  the  other  qualities. 
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Eleven  surveys  were  completed  and  returned.  Three  of 
the  qualities  (leadership,  judgment,  and  responsibility)  had 
consistently  high  ratings  and  were  selected  as  PC  dimensions. 
Seven  of  the  qualities  were  clearly  perceived  as  being 
inappropriate  PC  dimensions.  Appearance  and  cooperation  were 
selected  from  the  remaining  four  qualities  as  two  additional 
PC  dimensions  . 
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Phase  2 :  Form  development .  Three  research  forms  were 
developed  for  use  in  this  project.  The  PC  form  was  developed 
from  (a)  the  results  of  the  military  qualities  survey,  (b)  a 
combination  of  the  peer  nomination  and  peer  ranking  tech¬ 
niques  (e.g.,  Kane  &  Lawler,  1978),  and  (c)  the  psycho¬ 
physical  method  of  paired  comparisons  (Engen,  1971,  pp .  51- 
54) .  On  the  PC  form,  each  section  member  (a  class  is  divided 
into  two  sections)  is  required  to  nominate  and  rank  order 
five  peers  on  the  basis  of  their  potential  as  Army  aviation 
officers.  The  section  member  then  makes  paired  comparisons 
of  the  nominees  on  the  five  military  qualities  that  were 
selected  from  the  military  qualities  survey. 

PC  scores  are  computed  for  each  section  member  by  first 
summing  the  rank  score  (five  points  for  first  rank,  four 
points  for  second  rank,  .  .  .,  one  point  for  fifth  rank)  from 

each  nominating  peer.  The  summed  rank  scores  are  then  added 
to  the  number  of  favorable  comparisons  the  section  member 
received  on  each  military  quality.  The  total  is  then  divided 
by  the  maximum  possible  score  to  enable  direct  comparisons 
between  sections  with  unequal  numbers  of  students.  The 
resulting  PC  scores  could  range  from  0.0  (no  nominations)  to 
1.0  (ranked  first  by  all  peers  and  always  favorably  compared 
with  the  other  nominees) . 

A  faculty  advisor  rating  (FAR)  form  was  developed  to 
obtain  independent  evaluations  of  the  students'  potential  as 
Army  aviation  officers.  Each  AVNOAC  faculty  advisor  super¬ 
vises  approximately  six  students.  The  advisors  use  the  FAR 
to  estimate  where  each  of  their  students  would  rank  in  terms 
of  their  officer  potential  in  an  average  group  of  100 
captains.  Finally,  a  student  critique  form  was  developed  to 
ascertain  student  attitudes  toward  the  peer  comparison 
program.  The  students  are  asked  to  rate  the  fairness, 
utility,  aversiveness,  and  difficulty  of  various  aspects  of 
the  program.  They  are  also  asked  to  express  their  opinions 
about  the  implementation  of  the  program  and  to  offer 
recommendations  for  improving  the  program. 

Phase  3;  Experimental  administration  1.  Peer  compari¬ 
sons  were  collected  on  an  experimental  basis  (i.e.,  the  PC 
scores  were  not  used  to  select  honor  graduates)  from  Sections 
1  (n  =  41)  and  2  (r.  =  40)  of  AVNOAC  85-2  in  July  1985.  A 
second  set  of  PC  ratings  and  the  student  critiques  were 
collected  approximately  one  month  later.  The  faculty 
advisors  completed  the  FARs  immediately  after  graduation.  In 
addition,  the  final  academic  averages  (AVGs)  were  obtained 
from  the  School  Secretary's  office. 


The  results  of  the  first  administration  were  mixed.  The 
PC  scores  ranged  from  0.0  to  .92  in  Section  1  and  from  0.0  to 
.75  in  Section  2.  The  scores  indicate  a  high  consensus  among 
the  members  of  the  class  in  identifying  peers  with  the 
highest  potential  as  aviation  officers.  The  scores  for  the 
first  and  second  data  collections  were  highly  correlated 
(Section  1  =  .96  and  Section  2  =  .86),  indicating  the 
stability  of  the  ratings  over  time.  Combining  the  scores 
from  both  data  collections  in  both  sections,  the  class 
members  were  in  agreement  on  the  top  10%  and  the  lower  75%  of 
the  class. 

The  PC  scores  were  then  correlated  with  the  FARs  and 
AVGs .  For  Sections  1  and  2,  respectively,  the  PC  correla¬ 
tions  are  .45  and  .33  with  the  FAR,  and  .55  and  .29  with  the 
AVG.  These  correlations  are  sufficiently  high  to  show  an 
expected  relationship  between  observations  of  the  same 
individuals.  At  the  same  time,  the  correlations  are  suffi¬ 
ciently  low  to  indicate  that  the  PC  score  is  measuring  a 
unique  perspective  of  the  class  members.  The  correlations 
between  the  FAR  and  AVG  are  .74  and  .58  in  sections  1  and  2, 
respectively.  This  result  probably  indicates  that  the 
faculty  advisors  were  depending  upon  the  academic  average  as 
a  primary  source  of  information  in  making  their  ratings. 

Finally,  the  responses  to  the  PC  critique  were  tabu¬ 
lated.  The  overall  reaction  of  the  class  members  to  the  PC 
program  was  negative:  a  majority  indicated  that  the  PC  was 
very  biased,  slightly  or  not  at  all  useful,  and  slightly  or 
not  at  ail  predictive  of  future  performance.  Furthermore, 

72%  of  the  respondents  were  either  very  or  extremely 
unfavorable  toward  the  implementation  of  the  program.  The 
responses  to  the  other  critique  items  reflected  combinations 
of  positive,  negative,  and  neutral  attitudes,  without  any 
attitude  representing  a  majority  opinion. 

The  results  of  the  first  administration  indicated  that 
the  PC  technique  was  a  potentially  useful  procedure  for 
identifying  the  class  members  with  the  highest  potential  as 
Army  aviation  officers,  although  the  students  were  generally 
critical  of  its  use.  There  were,  however,  several  problems 
with  the  first  administration.  First,  the  students  were  not 
advised  i ^  advance  of  the  experimental  administration. 

Second,  a  concurrent  but  surreptitious  actentpL  by  the  class 
leaders  to  evaluate  the  section  members  was  discovered  jusu 
before  the  second  data  collection.  Both  of  these  problems 
may  have  affected  the  students'  attitudes  about  class 
evaluations.  Finally,  the  period  of  time  that  elapsed 
between  the  first  and  second  PC  administrations  was  too  short 
to  evaluate  the  stability  of  the  peer  assessments. 


Phase  3; _ Experimental  administration  2.  The  second 

experimental  administration  was  designed  as  a  replication  of 
the  first  administration,  with  the  following  changes: 

•  students  were  advised  in  advance  of  the  research, 

•  other  non-academic  evaluations  were  prohibited, 

•  three  months  elapsed  between  the  initial  and  final 
data  collection, 

•  the  military  quality  definitions  were  modified 
slightly, 

•  the  order  of  presentation  of  the  military  qualities 
and  nominee  pairs  was  completely  counterbalanced,  and 

•  a  new  roster  coding  system  was  instituted  to  protect 
student  privacy. 

The  PC  was  administered  to  4  9  students  in  Section  1  and 
51  students  in  Section  2  of  AVNOAC  86-1  in  December  1985.  in 
March  1986,  47  students  in  Section  1  and  49  students  in 
Section  2  completed  the  PC  ratings  and  student  critiques. 
Preliminary  analyses  indicated  an  overall  consensus  in  each 
section  on  the  members  having  the  highest  potential  as 
aviation  officers,  although  there  were  some  changes  in  the 
ratings  over  time.  There  was  a  longer  interval  between 
ratings  in  class  86-1  and  the  initial  ratings  may  have  been 
collected  before  peer  opinions  were  fully  formulated. 
Nonetheless,  the  correlations  between  the  initial  and  final 
ratings  are  .79  in  Section  1  and  .93  in  Section  2.  Similar 
to  class  85-2,  the  majority  of  class  86-1  were  opposed  to  the 
implementation  of  the  PC  program  in  the  AVNOAC . 


Further  analyses  were  not  conducted  pending  the  receipt 
of  missing  FAR  and  AVG  data.  In  November  1986,  the  missing 
AVG  data  was  finally  obtained  from  the  Academic  Records 
office,  but  the  Assistant  School  Secretary  terminated 
attempts  to  obtain  16  missing  FAR  ratings;  the  faculty 
advisors  who  had  not  submitted  complete  FARs  had  changed 
assignments  or  could  not  recall  their  advisees'  performance 
in  sufficient  detail  to  provide  the  ratings. 


During  June  and  July  1987,  the  data  from  both 
classes  were  prepared  for  analysis.  First,  the  PC 
forms  were  reviewed  for  completeness  and  accuracy  ( 
following  instructions  in  using  the  form) .  A  small 
age  of  the  ratings  were  deleted  from  the  analyses, 
the  PC  forms  were  rescored  by  Anacapa  Sciences,  Inc 
sonnei.  The  original  scores  had  been  tabulated  by 
in  the  School  Secretary's  office,  but  numerous  erro 
noted  during  the  rating  form  reviews.  In  addition, 
for  class  86-1  were  scored  so  that  split-half  analy 
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be  conducted  separately.  Finally,  the  data  were  entered  into 
a  new  computer  data  base. 


Work  Projected 

Two  activities  remain  to  be  completed  on  the  PC  project . 
First,  the  final  data  analyses  will  be  conducted  for 
Experimental  Administrations  1  and  2.  The  analyses  are 
designed  to: 

•  evaluate  the  internal  consistency  and  retest  reli¬ 
ability  of  the  PC  ratings, 

•  determine  the  relationship  between  the  PC  ratings  and 
other  evaluations, 

•  evaluate  the  internal  structure  of  the  PC  technique 
(e.g.,  are  ratings  on  five  military  qualities  needed), 

•  determine  student  reactions  to  the  PC  program,  and 

•  evaluate  various  methods  of  combining  the  PC  ratings 
with  AVGs  to  select  the  course  honor  graduates. 

Second,  a  report  and  a  briefing  will  be  prepared  to 
present  the  results  of  the  research  to  the  School  Secretary's 
office.  The  report  will  include  recommendations  on  the 
implementation  of  the  PC  program. 
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EVALUATION  OF  THE  AVIATION  RESOURCE  MANAGEMENT  SURVEY 

(ARMS)  CHECKLIST 

Dr.  John  W.  Ruffner,  Project  Director 


Background 

According  to  the  Army's  "total  force"  concept,  Reserve 
Component  (RC)  aviators  serving  in  the  U.S.  Army  Reserve 
(USAR)  and  the  Army  National  Guard  ( ARNG )  are  required  to 
train  to  the  same  standards  and  to  maintain  the  same  level  of 
flight  proficiency  as  aviators  serving  in  the  Active  Compo¬ 
nent  (AC) .  To  meet  these  requirements,  it  is  necessary  for 
the  managers  responsible  for  the  planning,  implementation, 
and  evaluation  of  RC  training  to  utilize  the  available 
resources  (e.g.,  aircraft,  training  time,  flying  hours, 
instructor  pilots)  as  efficiently  as  possible.  This  is 
particularly  important  since  the  amount  of  time  available  for 
training  RC  aviators  is  much  more  limited  than  for  AC 
aviators . 

One  of  the  ways  in  which  the  Army  assists  RC  training 
managers  in  meeting  these  requirements  is  through  the  use  of 
evaluation  visits  made  by  Aviation  Resource  Management  Survey 
(ARMS)  teams.  Forces  Command  (FORSCOM)  Regulation  350-3 
(1984)  states  that  the  general  purpose  of  the  ARMS  is  to 
"evaluate  the  management  of  unit  aviation  programs,  identify 
areas  requiring  additional  emphasis,  and  provide  staff  assis¬ 
tance  as  necessary." 

As  defined  by  FORSCOM,  the  ARMS  has  four  specific 
object ives : 

•  to  assist  commanders  in  identifying  strengths  and 
weaknesses  in  all  aviation  related  programs; 

•  to  assess  the  aviation  support  facility's  (ASF) 
ability  to  support  units  assigned  to  train  at  the 
facility  in  accomplishing  their  mobilization  mission; 

•  to  assess  the  aviation  unit's  capability  to  operate 
safely,  efficiently,  and  effectively,  and  to  maintain 
aviation  resources  separate  from  the  ASF;  and 

•  to  identify  systematic  problems  and  to  provide  assis¬ 
tance  when  the  capability  to  solve  the  problems  is 
beyond  the  installation  commander's  realm  of 

inf luence  . 

The  office  of  the  Deputy  Chief  of  Staff  for  Training 
(DCST)  in  each  of  the  five  Continental  U.S.  Armies  (CONUSAs) 
is  responsible  for  conducting  ARMS  evaluations.  According  to 


FORSCOM  Regulation  350-3  (1984),  an  ARMS  is  to  be  conducted 
at  least  once  a  year  for  each  USAR  facility  and  at  least  once 
every  two  years  for  each  ARNG  facility  within  the  CONUSA. 


Problem 

Each  CONUSA  has  its  own  procedure  for  carrying  out  the 
ARMS  evaluation  mission.  There  is  a  lack  of  standardization 
across  the  CONUSAs  in  (a)  the  functional  areas  (e.g.,  safety, 
standardization,  and  training)  that  are  evaluated,  (b)  the 
procedures  used  by  the  ARMS  teams  to  assess  the  facilities 
and  units,  and  (c)  the  standards  for  acceptable  performance. 

The  first  U.S.  Army  DCST,  Aviation  Division,  has 
developed  a  checklist  to  be  used  by  the  ARMS  team  during  its 
evaluation  visits.  The  checklist  originally  was  published  in 
October  1983,  and  subsequently  was  revised  in  August  1985,  as 
DA  Pamphlet  95-1,  Reserve  Component  Commander's  Guide  - 
Aviation  Standardization  and  Training  Program  Evaluation  and 
Aviation  Resource  Management  Survey.  The  checklist  draws 
heavily  from  two  sources:  (a)  FORSCOM  Form  14-1-R  Reserve 
Component  Aviation  Resource  Management  Ch.eckli.s.t  (1980),  and 
(b)  the  U.S.  Army  Safety  Center  Guide  to  Aviation  Resources 
M.a..aa,-g.am.e.n.t...-.£.Q.r_Aix..G£a.f.Ltli,,.s.h,ap,..,£r..e-y-e.n,t..i-O.Q  (1984),  but  does 
not  completely  replicate  either  document . 

The  First  Army  checklist  contains  approximately  650 
items  divided  into  the  following  11  major  functional  areas  of 
evaluation  : 

•  Aviation  Safety  Management, 

•  Facility/Unit  Operations, 

•  Aviation  Standardization  and  Training, 

•  Aircraft/Flightline  Operations, 

•  Aeromedical  Management, 

•  Aircraft  Crash  Rescue  and  Fire  Fighting, 

•  Petroleum,  Oil,  Lubricants  (POL)  Facilities  and 
Operations, 

•  Maintenance  Management, 

•  Aviation  Armament, 

•  Aviation  Life  Support  Equipment  (ALSE) ,  and 

•  Physical  Security. 

The  items  within  each  of  the  functional  areas  describe  a  con¬ 
dition  or  state  that  the  facility  or  the  units  must  meet. 

The  items  were  written  by  aviation  subject  matter  experts 
(SMEs)  who  are  knowledgeable  about  operational  requirements 


RC  support  facilities  must  support  and  about  mobilization 
mission  requirements  RC  units  must  fulfill. 

The  DCST,  First  U.S.  Army,  has  expressed  concern  about 
several  deficiencies  in  (a)  the  content  and  organization  of 
the  checklist,  (b)  the  manner  in  which  the  checklist  is  used 
to  evaluate  the  status  of  each  RC  facility  and  unit,  and  (c) 
the  quality  and  utility  of  feedback  provided  to  command 
personnel  following  the  evaluations.  Consequently,  during 
the  second  quarter  of  FY  1985,  the  DCST  requested  that  the 
U.S.  Army  Research  Institute  Aviation  Research  and  Develop¬ 
ment  Activity  (ARIARDA)  provide  research  support  to  evaluate 
and  revise  the  checklist.  Anacapa  Sciences,  Inc.,  began  work 
on  the  project  on  3  June  1985. 


Project  Objectives 

This  project  has  two  general  objectives:  (a)  to  perform 

a  systematic  evaluation  of  the  content  of  the  First  Army  ARMS 
Checklist,  the  procedures  used  to  administer  the  checklist, 
and  the  procedures  used  to  manage  information  from  the  ARMS 
visits;  and  (b)  to  make  recommendations  for  improvements  as 
necessary . 


Research  Approach 

A  preliminary  evaluation  of  the  ARMS  Checklist  indicated 
that  several  deficiencies  existed  in  the  checklist  and 
administrative  procedures: 

•  The  ARMS  Checklist  is  excessively  long  and  contains 
many  items  that  may  not  be  highly  related  to  mission 
success . 

•  The  Checklist  items  are  not  organized  so  that  an  eval¬ 
uator  can  proceed  through  the  evaluation  steps  in  an 
efficient  manner. 

•  The  items  are  not  identified  as  applying  specifically 
to  a  facility,  to  a  unit  or  to  both. 

•  It  is  difficult  to  detect  the  deficiencies  described 
in  many  items  during  the  time  allocated  for  an  ARMS 
evaluation . 

•  There  is  no  systematic  procedure  for  organizing  or 
managing  the  information  gathered  during  ARMS  evalua¬ 
tion  visits  . 

The  preliminary  evaluation  led  to  the  development  of 
three  criteria  for  determining  if  an  item  should  be  retained 
in  the  checklist.  Specifically,  an  item  should  be  retained 
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in  the  checklist  only  it  tne  attriouie  addressed  in  the  item 
is:  (a)  easily  detectable  during  an  ARMS  visit  (Detect¬ 
ability)  ,  (b)  important  for  judging  the  status  of  one  of  the 

functional  areas  (Importance) ,  and  (c)  critical  for  mission 
success  (Criticality) .  A  survey  questionnaire  was  developed 
to  assess  the  extent  to  which  the  checklist  items  meet  the 
three  criteria  for  a  support  facility  and  for  a  unit  .  A 
separate  version  of  the  questionnaire  was  developed  for  each 
of  the  functional  areas.  The  respondents  for  the  question¬ 
naire  were  aviators  and  aviation  technicians  from  First  Army 
National  Guard  and  Reserve  aviation  support  facilities  and 
aviation  units. 


Project  Status 

Work  Completed 

Pretesting  of  the  questionnaires  was  completed  in 
November  1985.  Following  pretest  and  revision,  the  question¬ 
naires  were  mailed  to  ARNG  and  USAR  facilities  in  the  First 
Army  area.  An  average  of  23  respondents  completed  a  ques¬ 
tionnaire  in  each  functional  area.  Responses  to  the  ques¬ 
tionnaires  were  entered  into  a  data  base,  verified,  and 
analyzed.  Preliminary  results  of  the  data  analyses  were 
briefed  to  staff  members  of  the  Aviation  Division,  DCST, 

First  U.S.  Army  in  March  1987. 

In  June  1987,  a  draft  technical  report  describing  the 
method  and  results  was  submitted  to  ARIARDA  for  formal  review 
(Ruffner  &  McAnulty,  1987) .  The  report  is  entitled  "An 
Evaluation  of  the  Aviation  Resource  Management  Survey  (ARMS) 
Checklist."  The  major  results  described  in  the  report  are 
summarized  in  the  following  paragraphs. 

The  respondents  rated  over  one-half  of  the  checklist 
items  as  moderately  high  on  Detectability  and  Importance  for 
both  a  facility  and  a  unit.  However,  nearly  all  the  items 
were  rated  low  on  Criticality.  The  low  Criticality  ratings 
suggest  that  most  of  the  deficiencies  described  in  the  check¬ 
list  items,  in  isolation,  would  not  prevent  an  RC  facility  or 
unit  from  accomplishing  its  mission.  The  results  also 
suggest  that  the  Facility  Detectability  and  Facility  Impor¬ 
tance  summary  scores,  but  not  the  Facility  Criticality 
summary  score,  should  be  used  to  determine  if  items  should  be 
retained  in  their  present  form,  revised,  or  deleted  from  the 
checklist.  In  addition,  the  results  suggest  that  a  single 
version  of  the  checklist  needs  to  be  developed,  with  each 
item  clearly  annotated  to  indicate  whether  it  applies  to  a 
facility  or  to  a  unit. 
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An  information  data  base  was  developed  as  part  of  the 
project.  The  data  base  can  be  used  to  summarize  (a)  the 
average  summary  scores  for  the  Detectability,  Importance,  and 
Criticality  of  the  checklist  items,  and  (b)  the  performance 
of  RC  units  on  specific  checklist  items  and  functional  areas. 
The  data  base  was  designed  to  enable  the  First  Army  ARMS  team 
to  reorganize  the  checklist  by  identifying  items  with  similar 
content  and  reference  publications,  and  to  utilize  the  data 
obtained  from  ARMS  visits  more  effectively  (e.g.,  to  identify 
commonly  occurring  deficiencies) . 


Delivery  of  the  report  completes  Anacapa 1 s  work  on  the 
project  unless  First  Army  requests  further  assistance. 
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-  aviation  standardization  training  program  evaluation 
and  aviation  resource  management  survey  (DA  Pamphlet  95- 
1) .  Fort  Meade,  MD :  Department  of  the  Army. 

Ruffner,  J.  W.,  &  McAnulty,  D.  M.  (1987).  An  evaluation  of 

the  aviation  resource  management  survey _ (ARMS). 

checklist:  Vol  s  I  and  II  (Technical  Report  ASI690-301- 

87  [&]).  Fort  Rucker,  AL :  Anacapa  Sciences,  Inc. 


U.S.  Army  Forces  Command  (1984)  . 


_nma  in 


(Regulation  350-3)  .  Fort  McPherson,  GA:  Department  of 
the  Army. 


Army  Forces  Command  (1980).  Reserve  component  aviation 


resource  management  survey  checklist  (FORSCOM  Form  14-1 
R) .  Fort  McPherson,  GA:  Department  of  the  Army. 


Army  Safety  Center  (1984) .  ( 


ement  for  aircraft  mish. 
Department  of  the  Army. 


Fort  Rucker 


SUMMARY  OF  ACTIVITIES  ON  THE  AH-64  REFORGER  PROJECT 
Dr.  D.  Michael  McAnulty,  Project  Director 

Background 

On  6  August  1987,  the  Commander  of  the  U.S.  Army 
Aviation  Center  (USAAVNC)  tasked  the  Directorate  of  Training 
and  Doctrine  (DOTD)  to  develop  a  survey  and  determine  the 
data  collector  requirements  needed  "to  resolve  the  AH-64 
front  seat/back  seat  issue."  According  to  the  tasking,  the 
survey  was  to  evaluate  the  following  three  options  and  to 
recommend  the  policy  to  be  implemented.  The  three  options 
are : 

•  no  restrictions  for  AH-64  pilots  switching  seats, 

•  pilots  changing  seats  with  the  approval  of  the  unit 
commander,  and 

•  pilots  changing  seats  with  the  approval  of  the 
USAAVNC . 

The  survey  information  and  other  data  were  to  be 
collected  from  squadrons  of  the  6th  Cavalry  Brigade  from  Fort 
Hood,  Texas,  at  the  REFORGER  exercise  in  the  Federal  Republic 
of  Germany  (FRG)  during  September  1987.  Upon  the  request  of 
DOTD,  the  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  agreed  to  develop  the  data 
collection  instruments  and  to  provide  three  data  collectors 
during  the  REFORGER  exercise  . 


Need 

A  series  of  meetings  was  held  with  representat ives  of 
several  organizations  at  the  USAAVNC  and  with  the  6th  Cavalry 
Brigade  to  define  the  research  issues  and  to  obtain  relevant 
documents.  Several  interrelated  training,  system  design,  and 
aviator  management  issues  were  identified  as  pertinent  to  the 
project.  In  essence,  however,  the  AH-64  front  seat/back  seat 
issue  involves  the  effectiveness  of  an  Aircrew  Training 
Manual  (Headquarters,  U.  S.  Army  Aviation  Center:  FC  1-214) 
policy  that  stipulates  that  AH-64  aviators,  with  certain 
exceptions,  are  to  be  designated  as  either  front  seat  or  back 
seat  pilots.  That  is,  most  of  the  AH-64  aviators  maintain 
currency  in  only  one  seat  and  receive  annual  familiarization 
training  (approximately  10%  of  their  annual  flight  training) 
in  the  opposite  seat.  The  aviators  must  complete  refresher 
training  before  switching  seat  assignments.  In  addition,  FC 
1-214  encourages  the  scheduling  of  fixed  crews. 
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Development  of  instruments  and  procedures.  The  three 
research  instruments  were  developed  and  produced  lor  use  in 
the  data  collection  effort.  The  data  collection  procedures 
were  developed  and  coordinated  with  representatives  of  the 
6th  Cavalry  Brigade.  Both  survey  forms  were  produced  in 
sufficient  quantities  to  collect  data  from  all  the  AH-64 
aviators  (1st  and  2nd  Squadrons  of  the  6th  Cavalry  Brigade) 
participating  in  REFORGER.  The  first  survey  was  scheduled  to 
be  administered  approximately  two  days  before  the  exercise 
and  the  second  questionnaire  was  scheduled  to  be  administered 
approximately  two  days  after  the  exercise.  Because  of  time 
and  resource  limitations,  the  project  staff  decided  to  inter¬ 
view  aviators  from  only  one  of  the  two  squadrons  partici¬ 
pating  in  the  REFORGER  exercise.  As  previously  indicated, 
the  interviews  were  scheduled  immediately  after  the  crews 
returned  from  their  latest  flights. 


The  pre-REFORGER  survey  contains  81  items  divided  into 
three  parts.  Part  A  consists  of  30  items  requesting  objec¬ 
tive  information  about  the  respondents'  personal,  military, 
and  aviation  training  background.  Part  B  consists  of  24 
items  requesting  objective  and  subjective  information  about 
the  respondents'  experiences  with  the  AH-64  optical  systems. 
Part  C  consists  of  27  items  requesting  information  about  the 
respondents'  opinions  and  preferences  related  to  their 
training  as  AH-64  aviators  and  the  policies  that  are  used  to 
manage  the  AH-64  units. 


The  post-REFORGER  survey  consists  of  25  items  divided 
into  two  parts.  Part  A  consists  of  15  items  that  request 
objective  information  about  the  respondents'  experiences 
during  the  REFORGER  exercise.  Part  B  consists  of  10  items 
that  request  the  respondents'  personal  opinions  about  the 
effects  of  the  REFORGER  experience  on  the  aviator's 
capabilities  and  the  effect  of  current  management  policies  on 
the  capabilities  of  the  unit  during  the  exercise.  Some  of 
the  subjective  items  are  the  same  on  the  pre-  and  post- 
REFORGER  surveys  to  determine  if  the  exercise  experience  had 
changed  the  respondents'  opinions  or  preferences. 


The  structured  interview  schedule  consists  of  66  items 
divided  into  five  sections.  The  first  section  contains  28 
items  requesting  basic  information  (e.g.,  aircraft  number, 
time  since  the  flight  ended)  about  the  flight  and  background 
information  about  the  two  crewmembers.  Many  of  the  back¬ 
ground  items  in  this  section  are  redundant  with  the 


pre-REFORGER  survey  and  would  not  be  asked  if  the  aviators 
participated  in  the  Phase  1  data  collection.  The  second 
section  consists  of  11  items  requesting  general  information 
(e.g.,  flight  time,  environmental  conditions,  mission  objec¬ 
tives)  about  the  most  recent  sortie  the  crew  had  flown. 

The  last  three  sections  consist  of  items  about  specific 
aspects  of  the  most  recent  sortie:  Section  3  consists  of  16 
items  about  the  PNVS  capabilities  during  the  sortie;  Section 
4  consists  of  8  items  about  the  crew  coordination  during  the 
sortie;  and  Section  5  consists  of  3  items  about  target 
engagements  during  the  sortie.  If  the  crew  had  flown  more 
than  one  sortie  since  the  last  debriefing,  the  items  in 
Sections  2  through  5  would  be  repeated  for  each  sortie  that 
had  been  flown,  if  time  permitted. 

Project  coordination.  By  4  September,  arrangements  were 
completed  for  the  research  team  to  depart  for  the  FRG  on  8 
September,  except  for  the  receipt  of  a  theatre  clearance  from 
the  7th  Army  Training  Command  (ATC) .  The  principal  problem 
in  obtaining  the  clearance  was  that  the  AH-64  fleet  had  been 
grounded  pending  the  investigation  of  a  recent  accident  and 
the  correction  of  any  contributing  mechanical  problems.  On 
11  September,  the  project  team  was  informed  that  the  data 
collection  effort  had  been  cancelled,  even  though  the 
grounding  order  had  just  been  rescinded. 

Product  delivery.  A  brief  summary  (McAnulty,  Kaempf,  & 
Blackwell,  1987)  of  the  project  activities  was  prepared  and 
submitted  to  ARIARDA  in  September  1987.  Copies  of  the  two 
surveys  and  the  structured  interview  items  were  appended  to 
the  summary.  The  summary  concluded  that  ARIARDA  should: 

•  conduct  no  further  work  on  the  project  until  addi¬ 
tional  instructions  are  received  from  the  USAAVNC, 

•  consider  other  methodologies  in  addition  to  the  survey 
and  interview  approaches  developed  for  REFORGER,  if 
further  research  on  the  front  seat/back  seat  issue  is 
requested,  and 

•  conduct  any  future  research  after  a  period  of  "normal 
training"  has  occurred  following  the  recision  of  the 
AH-64  grounding  order  and  completion  of  the  REFORGER 
exercise . 


Work  Projected 

No  further  work  is  planned  on  this  project.  If  further 
research  is  required  on  the  AH-64  front  seat/back  seat  issue, 
a  new  project  will  be  initiated. 


(1986,  May) .  Fort  Rucker,  AL :  Headquarters,  U.  S.  Army 
Aviation  Center. 
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EVALUATION  OF  NATIONAL  GUARD  AVIATOR  TRAINING  REQUIREMENTS 

Dr.  John  W.  Ruffner  and  Dr.  Sandra  M.  Szabo, 

Project  Directors 


Ra  ckcrround 

An  aviator  in  the  Army  National  Guard  (ARNG)  must  ful¬ 
fill  the  same  annual  training  requirements  as  an  aviator  in 
the  active  Army.  The  requirements  are  outlined  in  the 
Aircrew  Training  Manuals  (ATMs)  for  individual  training  and 
in  the  Army  Training  and  Evaluation  Program  (ARTEP)  manuals 
for  collective  unit /combined-arms  training.  Both  ATM  and 
ARTEP  requirements  have  changed  significantly  since  the  early 
1970s,  when  most  ARNG  aviators  presently  in  the  force  were 
originally  trained.  Moreover,  the  ARNG  aviation  fleet  has 
been  modernized  significantly  since  that  time  and  several 
additional  aviator  training  requirements  have  been  added. 

The  major  requirements  that  have  been  added  include  the 
following : 

•  instrument  qualification, 

•  nap-of-the-earth  (NOE)  qualification, 

•  unaided  night  tactical  training, 

•  night  vision  goggle  (NVG)  qualification, 

•  qualification  in  ARNG-specif ic  aircraft'  (e.g.,  CH-54, 
OH-6)  ,  and 

•  qualification  in  attack  helicopter  systems  (e.g.,  AH- 
1G,  AH- IS  [MC],  AH- IS  [Mod]). 

In  some  instances,  the  courses  necessary  to  meet  the 
additional  qualifications  are  no  longer  offered  by  the  U.S. 
Army  Aviation  Center  (USAAVNC)  at  Fort  Rucker,  Alabama;  in 
other  instances,  it  is  impractical  for  the  ARNG  aviators  to 
attend  the  courses  at  USAAVNC.  Therefore,  the  aviator  must 
obtain  the  training  necessary  to  meet  the  requirements  by 
using  National  Guard  support  personnel  and  facilities  during 
his  or  her  available  training  time. 

Despite  the  increase  in  the  number  and  complexity  of  the 
training  requirements  that  aviators  must  meet,  the  amount  of 
time  allocated  for  ARNG  aviation  training  has  remained  rela¬ 
tively  constant  since  the  early  1970s.  ARNG  aviators 
currently  must  accomplish  their  annual  training  requirements 
during  a  combination  of  the  three  types  of  training  periods 
described  below. 


A  UTA  consists  of  a  four-hour  training  period.  Forty- 
eight  UTAs  are  allocated  annually  to  each  ARNG  aviator.  Four 
UTAs  typically  are  scheduled  in  succession  to  constitute  a 
weekend  drill  period.  There  are  12  weekend  drill  periods 
during  the  year.  The  drill  periods  are  referred  to  as 
Multiple  Unit  Training  Assemblies  (MUTAs).  MUTAs  are  autho¬ 
rized  for  unit  training. 


Additional  Flight  Training  Periods _ (AFTPs ) 

An  AFTP  consists  of  a  four-hour  period  that  is  typically 
used  to  maintain  individual  crewmember  skills  and  to  accom¬ 
plish  the  hands-on  flight  components  of  the  Annual  Aviator 
Proficiency  and  Readiness  Test  (AAPART) .  ARNG  aviators  are 
authorized  24  AFTPs  per  calendar  year. 


Annual  Training  (AT) 

Annual  training  periods  typically  are  used  for  collec¬ 
tive  unit  and  combined-arms  training  employing  a  threat 
oriented  scenario.  Emphasis  is  placed  upon  unit  operations 
tasks  to  ensure  effective  internal  command,  control,  and 
communications,  as  well  as  external  coordination  with  higher 
headquarters  or  supported  units.  ARNG  aviators  are  autho¬ 
rized  15  days  of  AT.  In  addition,  another  type  of  training 
period,  a  Full  Time  Training  Duty  (FTTD )  day,  can  be 
scheduled  for  training  in  the  Synthetic  Flight  Training 
System  (SFTS)  and  for  special  missions.  FTTDs  are  scheduled 
and  approved  on  a  case-by-case  basis. 

The  training  requirements  that  the  ARNG  aviator  must 
meet  have  significantly  increased  over  the  last  ten  years, 
while  the  training  time  available  to  the  ARNG  aviator  has 
remained  constant.  In  addition  to  the  problem  of  limited 
training  time,  ARNG  aviators  experience  a  number  of  other 
factors  that  may  make  it  difficult  to  meet  the  training 
requirements.  Important  factors  other  than  limited  training 
time  include  the  following: 

•  the  ARNG  aviators'  commitments  to  their  civilian  job 
responsibilities, 

•  the  geographical  distances  between  the  ARNG  aviators' 
homes  or  places  of  work  and  the  aviation  facilities 
where  training  is  conducted,  and 


•  the  ARNG  aviators'  family  and  civic  responsibilities. 

These  factors  may  limit  the  ARNG  aviators'  capacity  to 
utilize  the  currently  allocated  time  in  an  efficient  and 
effective  manner. 

Difficulty  in  meeting  the  training  requirements  may 
seriously  reduce  the  aviators'  ability  to  achieve  and  main¬ 
tain  a  safe  level  of  aviator  proficiency.  An  unsafe  level  of 
proficiency,  in  turn,  may  cause  some  of  the  aviators  to  leave 
the  National  Guard.  The  potential  attrition  of  large  numbers 
of  ARNG  aviators  is  especially  critical  in  view  of  the  "aging 
of  the  force . "  When  the  older,  more  experienced  aviators 
leave  the  ARNG,  a  considerable  amount  of  experience  and 
expertise  will  be  lost.  Without  the  experience  and  expertise 
of  the  older  aviators,  unit  commanders  may  find  that  it  is 
more  difficult  for  the  younger,  less  experienced  ARNG 
aviators  to  meet  the  training  requirements. 

The  National  Guard  Bureau  (NGB)  recognizes  that  ARNG 
aviators  may  not  be  able  to  meet  the  training  requirements  in 
the  amount  of  time  currently  allocated.  To  understand 
existing  training  time  commitments,  the  NGB  requested  that 
the  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  ( ARIARDA)  provide  information  about  the 
ARNG  aviators'  ability  to  meet  the  training  requirements  in 
the  amount  of  time  that  is  presently  allocated.  The  NGB 
requested  that  ARIARDA  compile  the  information  on  seven  types 
of  ARNG  aviation  units: 

•  attack  helicopter  company/troop, 

•  air  cavalry  troop, 

•  combat  support  aviation  company, 

•  aviation  general  support  company, 

•  aerial  surveillance  aviation  company, 

•  air  ambulance  detachment,  and 

•  transportation  company. 

The  information  provided  by  the  aviators  in  these  units  will 
be  used  to  determine  if  additional  time  is  needed  to  meet 
ARNG  aviation  training  requirements. 


The  ARNG  aviation  training  requirements  research  has  six 
specific  objectives.  The  objectives  are  listed  below: 

•  determine  the  demographic  characteristics  of  the 
current  ARNG  aviator  force  (e.g.,  age,  years  of 
service,  number  of  flight  hours); 


•  determine  if  the  amount  of  time  that  is  spent  to  meet 
the  current  ARNG  aviation  training  requirements 
exceeds  the  amount  of  time  that  is  allocated  to  meet 
the  requirements; 

•  identify  factors  that  may  affect  the  ARNG  aviators' 
ability  to  utilize  the  allocated  time  to  meet  the 
requirements  (e.g.,  training  obstacles,  family 
influences,  time  commitments  to  civilian  job) ; 

•  determine  the  ARNG  aviators'  willingness  to  spend 
additional  time  to  meet  the  training  requirements; 

•  identify  factors  that  may  influence  the  ARNG  aviators' 
willingness  to  spend  additional  time  to  meet  the 
training  requirements  (e.g.,  attitudes,  civilian  job 
requirements,  family  influences,  training  obstacles); 

•  determine  the  current  career  intentions  of  ARNG 
aviators;  and 

•  identify  factors  that  may  influence  the  career  inten¬ 
tions  of  ARNG  aviators  (e.g.,  civilian  job  require¬ 
ments,  satisfaction  with  ARNG  job) . 

The  objectives  were  met  by  compiling  data  for  the  total  APNG 
aviator  force  and  for  each  of  the  seven  types  of  ARNG  avia¬ 
tion  units  specified  by  the  NGB . 
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The  research  approach  developed  to  meet  the  project 
:tives  has  three  phases.  Phase  1  consists  of  a  question- 
2  survey  of  all  ARNG  aviators.  The  questionnaire  is 
jned  to  collect  information  about  the  aviators'  percep- 
;  of  the  training  time  and  training  requirements.  In 
:ion,  the  questionnaire  requests  information  about  the 
:ors'  demographic  characteristics,  their  ARNG  career 
’lions,  and  their  willingness  to  spend  additional  time  to 
the  training  requirements.  Phase  2  consists  of  an 
rally  scannable  training  log  that  the  aviators  completed 
month  for  12  months.  The  training  log  is  designed  to 
de  information  about  the  actual  amount  of  time  the 
:ors  .trend  meeting  the  training  requirements.  Phase  2 
sts  of  t  he  consolidation  of  t.  he  data  obtained  in  Phase  1 
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Phase  1.  ARNG  Aviator  Questionnaire 

The  questionnaire  developed  during  Phase  1  consists  of 
the  three  parts  described  below. 

Part  1 . _ Current  training  requirements.  In  Part  1  of 

the  questionnaire,  aviators  rated  each  of  the  following 
variables  concerning  training  requirements: 

•  adequacy  of  the  current  training  requirements  for 
maintaining  a  safe  level  of  aviation  proficiency, 

•  adequacy  of  the  time  allocated  for  meeting  the 
training  requirements, 

•  willingness  to  spend  additional  paid  time  to  meet  the 
training  requirements,  and 

•  willingness  to  spend  additional  nonpaid  time  to  meet 
the  training  requirements. 

In  addition,  the  aviators  checked  the  factors  that  serve  as 
obstacles  to  meeting  the  training  requirements. 

Part  2.  Demographic  characteristics.  In  Part  2  of  the 
questionnaire,  the  aviators  provided  information  about  the 
following  demographic  characteristics: 

•  personal  characteristics  (e.g.,  age,  education); 

•  military  characteristics  (e.g.,  aircraft  qualifica¬ 
tion,  total  years  of  military  service) ; 

•  civilian  employment  (e.g.,  income,  supervisor's  atti¬ 
tude  toward  ARNG) ;  and 

•  family  factors  (e.g.,  employment  of  spouse,  family 
attitudes  toward  ARNG). 

Part  3.  National  Guard  career  intentions.  Part  3  of 
the  questionnaire  required  the  aviators  to  provide  informa¬ 
tion  about  the  following  aspects  of  their  career  intentions: 

•  intentions  to  stay  in  or  leave  the  ARNG, 

•  factors  influencing  the  intention  to  remain  in  or 
leave  the  ARNG, 

•  satisfaction  with  the  ARNG,  and 

•  general  comments  about  the  ARNG. 

It.,  s  e  _  /.  ... .  _AP  N 3  Aviator  Train  i  ng  Lo  g 

•  : .  : . . •  •  2  is  designed  to  provide  <A  'votive  inf  rr.a 

•  :  the  amount  of  time  that  is  necessary  ft;  no" 

■•so'  ARNG  aviation  training  re  so  :  ■  es  :  a  •  e  :  f  i 
u.  :  '  :  •  i  .  !y  .  n  r.r.ai  ccrnpuf  e  r  -  s  oo  r  •  ■■  i  da'  a  tad  io  -t  ic 
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(training  log)  was  developed  to  enable  the  ARNG  aviators  to 
report  hours  spent  on  flying  and  nonflying  training 
activities  during  different  types  of  training  periods.  The 
aviators  reported  the  amount  of  time  spent  on  each  of  the 
following  flying  activities: 

•  meeting  ATM  minimum  iteration  requirements  and  check- 
rides  not  as  part  of  ARTEP  training  (combined  arms/ 
collective) , 

•  meeting  ATM  minimum  iteration  requirements  during 
ARTEP  training, 

•  meeting  ARTEP  training  requirements  exclusive  of  ATM 
minimum  iteration  requirements, 

•  inflight  training  and/or  evaluation  of  other  aviators 
exclusive  of  ATM  minimum  iteration  requirements,  and 

•  performing  miscellaneous  flight  activities  exclusive 
of  ATM  minimum  iteration  requirements. 

The  aviators  also  reported  the  amount  of  time  spent  on 
each  of  the  following  nonflying  activities: 

•  performing  requireu  additional  duties  (e.g.,  supply 
officer,  motor  officer,  administrative  duties); 

•  completing  and  administering  military  education, 
common  soldier  skills,  and  career  development  training 
(e.g.,  correspondence  courses,  academic  aspects  of 
aviation  qualifications)  ; 

•  performing  preflight  and  postflight  tasks  (e.g.,  pre¬ 
postflight,  weather/mission  brief,  flight  records) ; 

•  preparing  for,  undergoing,  and  administering  oral  and 
written  nonflying  aviation  evaluations  (e.g.,  annual 
written  exam,  -10  test,  flight  physicals,  checkrides), 
and 

•  performing  miscellaneous  nonflyinq  activities  (e.g, 
crew  rest,  dead  time,  inspections,  meals,  formations) . 

The  aviators  reported  the  amount  of  time  spent  on  each 
of  t  fir.*  activities  described  above  during  the  following  types 
j'i  '  r  lining  periods: 

•  Unit  Training  Assembly, 

•  Additional  Flight  Training  Period, 

•  Fu I !  Time  i raining  Duty, 

•  Annua  1  Training, 

•  Y o a r  V o u n  d  Annual  Training, 

•  Add  i  l  i  or:a  1  Training  Assembly,  and 

•  bp  i  i  t  Ur,  i  t  Tra  i  u  i  nq  As.semb  1  y  . 

i n  a  id : *  ion,  aviators  reported  the  amount  of  t i me  spent  on  a 
r.r  r/  s*  a*  us  at  t  he  Mat  i  or. a  ]  Guard  f  no  i  1  ity  and  on  a  nonj  ay 
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status  away  from  the  National  Guard  facility  (e.g.,  home, 
office) . 


Data  obtained  from  the  questionnaire  and  the  training 
log  were  consolidated  during  Phase  3.  Analysis  of  the 
consolidated  data  provided  diagnostic  information  about  the 
aviators'  utilization  of  current  training  time  and  their  need 
for  additional  training  time. 


Pretesting  of  the  questionnaire  was  completed  in 
September  1983.  Following  revision  of  the  questionnaire  to 
incorporate  feedback  received  during  the  pretest,  the  ques¬ 
tionnaires  were  sent  to  the  ARNG  facilities.  A  total  of 
3,640  questionnaires,  representing  77%  of  the  ARNG  aviator 
population,  was  completed  and  returned  by  the  31  July  1984 
cutoff  date.  Data  from  the  questionnaires  were  entered  into 
a  data  base  and  verified.  Preliminary  results  of  the  data 
analyses  were  briefed  to  the  National  Guard  Bureau  and  to  BG 
Richard  Dean,  Deputy  Commander  of  the  Army  National  Guard,  in 
May  1984;  final  results  were  briefed  in  September  1984. 

In  June  1985,  a  draft  technical  report  describing  the 
method  and  results  of  the  questionnaire  was  submitted  to 
ARIARDA  for  formal  review.  A  coordination  copy  of  the  draft 
report  was  sent  to  the  NGB .  The  report  is  entitled  "An 
Evaluation  of  the  Training  Requirements  of  Army  National 
Guard  Aviators.  Phase  1:  Analysis  of  Questionnaire  Data" 
(Szabo,  Ruffner,  Cross,  &  Sanders,  1986)  The  report  was 
published  as  ARI  Technical  Report  730  in  November  1986.  The 
major  results  described  in  the  report  are  summarized  in  the 
paragraphs  that  follow. 

Demographic  characteristics.  Fifty-five  percent  of  the 
aviators  have  at  least  a  four-year  college  degree.  The 
aviators  typically  have  professional/technical  civilian  jons 
and  earn  a  median  civilian  income  of  $32,500.  The  aviators 
spend  a  median  of  50  hours  per  week  on  their  civilian  jobs. 

The  ARNG  aviators  have  attained  a  high  level  of  military 
experience.  Eighty  percent  of  the  aviators  have  some  type  of 
prior  military  experience  upon  entering  the  National  Guard. 
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The  aviators  have  a  median  of  14  years  of  total  military 
experience;  12  of  these  years  have  typically  been  spent  on 
flight  status.  During  their  time  in  the  military,  the 
aviators  have  logged  a  median  of  2,000  total  flight  hours. 

Career  intentions.  Approximately  25%  of  the  aviators 
have  completed  between  15  and  20  years  of  service  and,  conse¬ 
quently,  will  be  eligible  for  retirement  within  the  next  five 
years.  However,  only  38%  of  the  total  force  of  aviators 
indicate  that  they  plan  to  leave  the  ARNG  as  soon  as  they 
reach  20-year  retirement  eligibility;  52%  indicate  that  they 
plan  to  remain  until  30-year  retirement. 

The  three  most  important  reasons  for  both  joining  and 
remaining  in  the  National  Guard  are  the  opportunity  to  fly, 
pay,  and  retirement  benefits.  Pay  and  retirement  benefits 
are  somewhat  more  important  reasons  for  remaining  in  the 
National  Guard  than  they  are  for  initially  joining  the 
National  Guard.  The  factor  that  is  most  likely  to  influence 
the  aviators  to  leave  the  National  Guard  is  loss  of  flight 
status.  Unrealistic  training  goals  and  administrative 
details  and  politics  were  also  cited  by  the  majority  of  the 
aviators  as  reasons  for  possibly  leaving  the  ARNG. 

Training  requirements.  ARNG  aviators  judge  the  training 
time  to  be  inadequate.  The  time  is  particularly  inadequate 
for  meeting  night  vision  goggle  (NVG) ,  unaided  night  tacti¬ 
cal,  and  tactical/special  requirements;  furthermore,  the 
aviators  judge  that  these  requirements  are  inadequate  for 
maintaining  a  safe  level  of  aviator  proficiency. 

The  aviators  judge  the  training  time  to  be  marginally 
adequate  for  meeting  all  additional  military  requirements 
except  inflight  evaluation/training,  for  which  the  training 
time  is  judged  to  be  inadequate.  All  of  the  additional 
military  requirements  are  judged  to  be  only  marginally 
adequate  for  maintaining  a  safe  level  of  aviator  proficiency. 

The  aviators  are  very  willing  to  spend  additional  paid 
time  to  meet  all  the  continuation  training  requirements  and 
the  additional  military  requirements  that  are  related  to 
career  progression  and  aviation.  The  aviators  are  very 
unwilling  to  spend  additional  nonpaid  time  to  meet  any  of  the 
training  requirements. 

Obstacles  to  training.  The  major  obstacles  that-  ARNG 
aviators  encounter  in  meeting  the  continuation  training 
requirements  are  an  insufficient  number  of  flight  hours  and 
the  unavailability  of  instructor  pilots  (IPs) .  The  major 
obstacle  to  meeting  additional  military  requirements  is  an 
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insufficient  amount  of  personal  time.  The  requirement  whose 
accomplishment  is  impeded  most  by  training  obstacles  is  NVG 
training;  unavailability  of  equipment  is  the  major  obstacle 
to  meeting  the  requirement.  In  addition,  unavailability  of 
aircraft  and  unavailability  of  training  support  areas  are 
obstacles  to  meeting  specific  requirements  in  specific  types 
of  units . 


Work  Completed; _ Phase  2  and  Phase  3 

Pretesting  of  the  Phase  2  training  log  was  conducted  in 
September  1983,  in  conjunction  with  pretesting  of  the  Phase  1 
questionnaire.  The  training  logs  were  mailed  to  the  ARNG 
facilities  in  March  1984.  The  aviators  completed  the  logs 
each  month  from  June  1984  through  May  1985.  A  total  of  1,081 
aviators  completed  training  logs  for  at  least  10  months 
during  the  12-month  data  collection  period.  Missing  data 
were  estimated  for  aviators  who  had  completed  training  logs 
for  only  10  or  11  months.  Data  from  the  training  log  were 
entered  into  a  data  base  and  verified.  Results  of  the  data 
analysis  ware  briefed  to  LTC  Richard  Braman,  Standardization 
and  Training  Officer,  Aviation  Division,  National  Guard 
Bureau  in  March  1987. 

In  October  1986,  a  draft  technical  report  describing  the 
method  and  results  of  the  training  log  project  v.as  submitted 
to  ARIARDA  for  formal  review.  The  report  is  entitled  "An 
Evaluation  of  Army  National  Guard  Aviator  Training  Time 
Utilization"  (Ruffner  &  Szabo,  1986)  The  major  results 
described  in  the  report  are  summarized  in  the  paragraphs  that 
follow . 

Utilization  of  time.  The  principal  finding  from  Phase  2 
is  that  the  median  total  amount  of  time  ARNG  aviators  spent 
meeting  their  requirements  (546  hours)  exceeds  the  amount  of 
time  they  are  allocated  (428  hours) .  Forty-eight  percent  of 
the  additional  hours  were  spent  while  in  a  nonpaid  status  at 
a  location  away  from  the  aviation  support  facility.  Thirty- 
four  percent  of  the  hours  were  spent  during  AT.  The 
remaining  time  was  spent  in  UTAs  and  AFTPs . 


Twenty  percent  of  the  total  training  time  was  spent 
meeting  f 1 ight /s imulat or  training  requirements.  Thirty 
percent  of  the  flight/simulator  hours  was  spent  in  UTAs,  401 
in  AFTPs,  and  30%  in  AT.  Over  two-thirds  of  the  flight/ 
simulator  hours  in  UTAs,  AFTPs,  and  AT  were  spent  meeting  ATM 
requ i rement  s . 


Eighty  percent  of  the  total  time  was  spent  meeting  non¬ 
flying  training  requirements.  Forty  percent  of  the  nonflying 
hours  was  spent  in  UTAs,  10%  in  AFTPs,  and  35%  in  AT.  The 
remaining  15%  of  the  nonflying  hours  was  spent  meeting 
training  requirements  while  in  a  nonpaid  status  at  a  location 
other  than  the  aviation  facility.  Sixty  percent  of  the  non¬ 
flying  hours  in  UTAs  was  spent  on  additional  nonflying 
requirements  and  miscellaneous  nontraining  duties.  Sixty 
percent  of  the  nonflying  hours  in  AFTPs  was  spent  on  pre-  and 
postflight  activities.  Forty  percent  of  the  nonflying  hours 
during  AT  was  spent  on  additional  nonflying  duties. 

Factors  influencing  the  utilization  of  training  time. 
Several  demographic  and  rating- scale  variables  from  the 
Phase  1  questionnaire  were  examined  in  an  attempt  to  deter¬ 
mine  the  factors  that  influence  the  aviators'  utilization  of 
training  time.  Of  the  variables  examined,  statistically 
significant  differences  were  found  only  between  warrant 
officers  and  commissioned  officers.  Commissioned  officers 
logged  significantly  fewer  f light/simulator  hours  in  UTAs  and 
significantly  more  f light/simulator  hours  in  AFTPs  than  did 
warrant  officers.  In  addition,  commissioned  officers  spent 
significantly  more  nonflying  time  during  UTAs  and  ATs . 


Operational  Implications 

The  results  indicate  that  the  actual  amount  of  time  that 
ARNG  aviators  spend  meeting  aviation  training  requirements  is 
greater  than  the  amount  of  time  currently  allocated.  Most  of 
the  additional  time  is  spent  on  nonflying  training  activities 
conducted  during  AT  and  on  military  education  activities 
conducted,  without  pay,  away  from  the  training  facility. 
Therefore,  an  increase  in  the  total  amount  of  time  allocated 
for  ARNG  aviation  training  activities  may  be  necessar"  With 
few  exceptions,  the  results  of  the  training  log  survey  agree 
with  the  aviators'  perceptions  of  the  adequacy  of  the 
training  time,  as  reported  in  the  Phase  1  questionnaire 
survey . 

Work  Projected 

Delivery  of  the  report  comDletes  Anacapa ' s  work  on  the 
project  unless  the  ARNG  requests  additional  assistance. 
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DETERMINATION  OF  ARMY  RESERVE  COMPONENT 
TRAINING  REQUIREMENTS 

Dr.  John  W.  Ruffner,  Project  Director 


Background 

Anacapa  Sciences,  Inc.,  and  U.S.  Army  Research  Institute 
Aviation  Research  and  Development  Activity  (ARIARDA)  person¬ 
nel  recently  have  conducted  research  to  determine  the  ade¬ 
quacy  of  time  allocated  to  meet  Army  National  Guard  (ARNG) 
training  requirements  and  to  determine  the  military  and 
civilian  demographic  characteristics  of  ARNG  aviators.  The 
results  of  the  ARNG  research  are  reported  in  detail  by  Szabo, 
Ruffner,  Cross,  and  Sanders  (1986)  and  Ruffner  and  Szabo 
(1986),  and  are  summarized  in  this  report  in  the  section 
entitled  "Evaluation  of  National  Guard  Aviator  Training 
Requirements"  (pp.  69  -  79). 

Like  their  counterparts  in  the  ARNG,  aviators  in  the 
U.S.  Army  Reserve  (USAR)  Component  must  meet  the  same  annual 
training  requirements  as  aviators  in  the  active  Army.  The 
types  of  information  obtained  during  the  ARNG  study  are 
equally  valuable  for  addressing  training  and  personnel 
management  issues  in  the  USAR.  Therefore,  the  First  Army 
Deputy  Chief  of  Staff  -  Training  (DCST)  requested  that 
ARIARDA  and  Anacapa  obtain  demographic  data  and  information 
about  the  adequacy  of  training  requirements  and  training  time 
for  USAR  aviators  in  the  First  Army  area. 


Project  Objectives 

The  USAR  aviation  training  requirements  research  has  six 
specific  objectives.  The  objectives  are  listed  below: 

•  determine  the  demographic  characteristics  of  the 
current  First  Army  USAR  aviator  force  (e.g.,  age, 
years  of  service,  number  of  flight  hours); 

•  determine  the  current  career  intentions  of  First  Army 
USAR  aviators; 

•  identify  the  factors  influencing  First  Army  aviators' 
decision  to  join,  remain  in,  and  possibly  leave  the 
USAR; 

•  determine  if  the  aviators  consider  the  amount  of  time 
allocated  for  training  to  be  adequate  for  meeting  the 


•  determine  the  aviators'  willingness  to  spend 
additional  time  to  meet  the  training  requirements;  and 

•  identify  the  factors  that  may  affect  the  First  Army 
aviators'  ability  to  utilize  the  allocated  time  to 
meet  the  training  requirements  (i.e.,  training 
obstacles) . 


The  research  approach  adopted  for  this  project  is 
similar  to  that  used  for  the  ARNG  project.  The  approach  is 
described  in  detail  in  Szabo  et  al .  (1986)  and  summarized  on 

pages  54-64  in  this  report  (Ruffner,  1987) .  Briefly,  USAR 
aviators  completed  a  questionnaire  developed  to  assess  demo¬ 
graphic  variables,  adequacy  of  current  training  requirements, 
adequacy  of  the  time  allocated  to  meet  the  requirements, 
willingness  to  spend  additional  time  to  meet  the  require¬ 
ments,  and  obstacles  to  training. 


Work  began  on  the  project  in  June,  1985.  The  aviator 
questionnaire  that  was  used  in  the  ARNG  study  was  modified 
slightly  to  reflect  minor  differences  that  exist  between  the 
ARNG  and  USAR  (e.g.,  types  of  units  and  aircraft) .  In  addi¬ 
tion,  a  few  items  were  added  to  the  USAR  version  of  the 
questionnaire  to  obtain  information  of  interest  to  the  First 
Army  DCST  (e.g.,  extent  of  simulator  utilization) .  The  USAR 
aviator  questionnaire  was  pretested  in  November  1985.  The 
feedback  obtained  during  the  pretest  visits  was  used  to 
revise  the  questionnaire. 

The  questionnaire  was  distributed  to  the  aviators  in 
March  1986.  A  total  of  139  questionnaires,  representing  561 
of  the  USAR  aviators  in  the  First  Army  area,  was  completed 
and  returned  by  the  30  June  1986  cutoff  date.  Data  from  the 
questionnaires  were  entered  into  a  data  base  and  verified. 
Descriptive  statistics  (i.e.,  means,  standard  deviations, 
medians,  and  percentages)  were  generated  for  each  question¬ 
naire  item  for  the  total  sample  and  for  subsamples  of 
commissioned  officers  and  warrant  officers. 

A  written  summary  of  the  data  analysis  results  was 
submitted  to  ARIARDA  in  September,  1987.  The  major  results 
of  the  questionnaire  data  analysis  are  the  following: 


•  First  Army  USAR  aviators  have  somewhat  lower 
experience  levels  (e.g.,  flight  hours,  time  in 
service,  percent  with  combat  experience)  than  aviators 
in  the  ARNG. 

•  Similar  to  ARNG  aviators.  First  Army  USAR  aviators  are 
generally  satisfied  with  their  civilian  and  USAR  jobs 
and  generally  intend  to  stay  in  the  USAR  until  they 
are  eligible  for  at  least  a  20-year  retirement. 

Factors  that  influence  aviators  to  remain  in  the  USAR 
include  the  opportunity  to  fly,  pay,  and  retirement 
benefits.  Factors  that  may  influence  aviators  to 
leave  the  USAR  include  administrative  details  and 
politics,  unrealistic  training  goals  for  the  time  and 
resources  allocated,  and  loss  of  flight  status. 

•  First  Army  USAR  aviators  generally  rate  the  amount  of 
time  available  to  meet  continuation  training  require¬ 
ments  as  inadequate  and  are  willing  to  spend  addi¬ 
tional  paid  time  to  meet  the  requirements.  This 
finding  is  consistent  with  the  results  from  the  ARNG 
aviation  survey. 

•  Similar  to  the  ARNG  aviators,  First  Army  USAR  aviators 
judged  that  the  lack  of  instructor  pilots,  flight 
hours,  training  areas,  and  aircraft  are  the  ma]or 
obstacles  to  meeting  continuation  training  require¬ 
ments,  and  that  an  insufficient  amount  of  personal 
time  is  a  major  obstacle  to  meeting  additional 
military  requirements.  In  contrast  to  ARNG  aviators, 
First  Army  USAR  aviators  judged  that  the  unavail¬ 
ability  of  aircraft  is  an  obstacle  to  meeting  most  of 
the  continuation  training  requirements.  In  general, 
the  unavailability  of  resources  appears  to  be  a  more 
serious  problem  for  First  Army  USAR  aviators  than  it 
is  for  ARNG  aviators. 


Work  Projected 

Delivery  of  the  written  summary  of  the  data  analysis 
results  completed  Anacapa's  work  on  this  project  unless  First 
Army  requests  additional  assistance. 
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STUDY  OF  STAFF  AVIATOR  SKILL  DECAY/REACQUISITION 
Mr.  Carl  R.  Bierbaum,  Project  Director 


Background 

All  staff  aviators  stationed  at  the  U.S.  Army  Aviation 
Center  (USAAVNC)  were  prohibited  from  flying  from  March  to 
October  of  1986  to  conserve  limited  flying  resources.  The 
prohibition  was  lifted  in  October  1986,  reinstituted  shortly 
thereafter,  and  continued  until  February  1987.  As  a  conse¬ 
quence,  a  large  population  of  staff  aviators  located  at 
USAAVNC  had  not  flown  for  10  months  or  longer.  When  the 
prohibition  was  lifted  in  February,  Army  aviation  officials 
directed  that  all  staff  aviators  (a)  be  administered  a 
commander's  evaluation  checkride,  and  (b)  be  trained  to 
proficiency  by  instructor  pilots  (IPs)  assigned  to  the 
Directorate  of  Plans,  Training,  Mobilization,  and  Security 
(DPT),  Aviation  Division,  ATM  Branch. 

The  extended  period  of  grounding  and  subsequent  requali¬ 
fication  of  staff  aviators  was  recognized  by  *-he  A^my 
Research  Institute  Aviation  Research  and  Development  Activity 
(AR1ARDA)  as  an  opportunity  to  gather  valuable  data  on  the 
decay  and  reacquisition  of  flying  skills.  On  9  February 
1987,  ARIARDA  submitted  a  written  request  to  DPT,  Aviation 
Division,  asking  permission  to  compile  performance  data  on 
staff  aviators  during  both  the  commander's  evaluation  and 
subsequent  training  flights.  DPT  agreed  that  the  effort  was 
worthwhile  and  pledged  to  provide  the  requisite  support. 


Need 

Although  a  previous  study  of  Army  aviators  was  conducted 
to  determine  skill  decay  and  reacquisition  after  six  months 
of  nonflying  (Ruffner  &  Bickley,  1983),  and  a  study  was 
conducted  of  National  Guard/Reserve  aviators  who  had  not 
flown  for  an  extended  period  of  time  (up  to  five  years)  (Wick 
e~  al.  1984),  a  study  had  not  been  done  of  aviators  who  have 
not  flown  for  a  period  of  10-13  months.  The  prohibition  of 
staff  aviator  flying  for  the  period  from  March  1986  to 
February  1987  thus  provided  an  excellent  opportunity  to  fill 
the  gap  between  the  previous  studies. 
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The  primary  objective  of  the  staff  aviator  skill  decay- 
reacquisition  study  was  to  determine  the  impact  on  aviator 
proficiency  of  not  flying  for  a  period  of  10-13  months. 
Specifically,  the  research  was  designed  to: 

•  determine  the  degree  of  skill  decay  by  evaluating  the 
aviators'  level  of  proficiency  during  the  commander's 
evaluation  checkrides,  and 

•  determine  the  amount  of  flight  time  required  for 
reacquisition  of  skills  that  had  decayed  below 
acceptable  standards. 


The  research  approach  for  meeting  the  project  objectives 
was  conducted  in  three  phases.  In  Phase  1,  Anacapa  Sciences, 
Inc.,  researchers  developed  the  test  procedures  and  the 
necessary  data  collection  instruments  for  conducting  the 
study.  During  Phase  2,  evaluators  monitored  the  commander's 
evaluation  checkrides  to  collect  data  and  administered  a 
staff  aviator  questionnaire  during  postflight  debriefing.  In 
Phase  3,  the  data  collected  during  Phase  2  were  analyzed  and 
reported . 


A  preliminary  planning  meeting  was  held  with  personnel 
from  DPT,  Aviation  Division,  ATM  Branch;  ARIARDA;  and  Anacapa 
to  discuss  the  staff  aviator  study.  The  methods  and 
procedures  agreed  upon  are  listed  below: 

•  DPT  personnel  would  furnish  a  list  of  the  tasks  to  be 
evaluated  during  the  staff  aviators'  commander's 
evaluation  checkrides, 

•  Anacapa  personnel  would  develop  a  12-point  rating 
scale  with  descriptive  verbal  anchors  to  standardize 
aviator  performance  ratings  during  the  commander's 

e  v  a  !  u  a  t  i  o  n  c  h  e  c  k  r  i  d  e  n  , 

•  ARIARDA  personnel  would  designate  individuals  as 
observers  to  collect  data  during  the  commander's 
evaluation  checkrides  and  to  administer  a 
Questionnaire  for  personal  data,  and 
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•  DPT  personnel  would  notify  Anacapa  when  commander  s 
evaluation  checkrides  were  scheduled  for  aviators  who 
had  not  flown  for  10-13  months. 

Anacapa  personnel  then  developed  the  three  data  collection 
instruments  described  in  the  following  paragraphs. 

Commander's  evaluation  gradeslip.  The  commander's 
evaluation  gradeslip  was  developed  in  coordination  with  the 
DPT  IPs  to  evaluate  the  initial  performance  levels  of  the 
aviators  who  had  not  flown  for  10-13  months.  The  gradeslip 
was  divided  into  24  discrete  tasks  to  be  performed  during  the 
evaluation  flight.  The  gradeslip  format  required  that  the 
evaluator  record  (a)  the  order  in  which  the  tasks  were 
performed,  and  (b)  the  numerical  performance  rating  for  each 
task  on  a  12-point  scale. 

Practice  iteration  data  sheet.  The  practice  iteration 
data  sheet  was  developed  to  evaluate  the  flights  required  to 
retrain  aviators  who  did  not  perform  the  commander's 
evaluation  checkride  tasks  to  established  standards.  The 
data  sheet  also  was  divided  into  24  discrete  tasks  with 
columns  for  scoring  each  training  iteration  required  until 
the  aviator  successfully  performed  the  task  to  established 
standards . 

Staff  aviator  questionnaire.  The  staff  aviator  ques¬ 
tionnaire  was  developed  to  obtain  personal  information  from 
each  aviator.  Questionnaire  items  were  designed  to  collect 
information  about: 

•  previous  flight  experience, 

•  recent  academic  study  of  flying  material,  and 

•  simulator  training  during  the  previous  year. 


The  data  collection  commenced  on  3  March 
completed  on  12  August  1987.  All  IPs  administ 
commander's  evaluation  checkride  were  briefed 
che  12-point  rating  scale  developed  by  Anacapa 
Observers  were  designated  for  each  flight  to  r 
workload  and  to  facilitate  standardization  of 
collection.  The  observers  monitored  the  task 
and  recorded  the  numerical  performance  rating 
IP.  The  same  procedure  was  followed  for  addit 
iteration  flights.  During  postflight  debrief i 
aviators  were  requested  to  complete  the  staff 
questionnaire . 
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All  commander's  evaluation  checkrides  that  met  the 
criteria  for  the  study  were  observed  and  data  collection 
forms  were  completed.  Although  there  had  appeared  to  be  a 
large  number  of  aviators  who  had  not  flown  for  a  period  of 
one  year,  the  total  number  of  data  collection  flights  was 
only  18.  Only  2  of  the  18  aviators  required  an  additional 
practice  iteration  flight,  averaging  1.5  hours.  The  16 
aviators  who  successfully  completed  the  commander's  evalua¬ 
tion  checkride  required  an  average  of  1.6  hours  to  complete 
the  checkride.  The  number  of  data  collection  flights 
completed  did  not  provide  sufficient  data  to  perform  a 
quantitative  data  analysis  that  would  provide  conclusive 
results;  therefore,  the  study  was  terminated.  All  data 
collection  forms  have  been  submitted  to  ARIARDA. 
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DGFS  AVIATION  AMMUNITION  AND  GUNNERY  SURVEY 
Dr.  D.  Michael  McAnulty,  Project  Director 


Background 

In  January  1987,  the  Department  of  Gunnery  and  Flight 
Systems  (DGFS)  at  the  U.S.  Army  Aviation  Center  (USAAVNC) , 
Fort  Rucker,  Alabama,  requested  that  the  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
(ARIARDA)  provide  research  support  in  the  development  of  an 
aviation  ammunition  and  gunnery  (A&G)  survey.  The  A&G  survey 
was  scheduled  to  be  distributed  worldwide  to  approximately 
170  Army  and  National  Guard  aviation  units.  ARIARDA  agreed 
to  assist  in  designing  and  pretesting  the  survey;  to  develop 
the  data  entry,  verification,  and  analysis  programs;  to 
conduct  the  required  data  analyses;  and  to  assist  in  the 
preparation  of  a  technical  report,  if  needed.  All  other 
project  activities  (e.g.,  administrative  coordination, 
pretesting,  data  collection,  data  entry)  were  to  remain  the 
responsibility  of  the  DGFS  study  group. 


Problem 

The  survey  research  is  designed  to  provide  an  empirical 
data  base  for  addressing  three  major  problem  areas.  First, 
the  increasing  cost  of  ammunition  and  the  competition  for 
Department  of  Defense  funds  have  created  pressure  to  reduce 
the  annual  allocation  of  ammunition  for  Army  aviation  gunnery 
training.  The  research  is  intended  to  document  the  current 
utilization  of  ammunition  in  aviation  gunnery  training  and  to 
compile  estimates  of  the  amount  of  ammunition  required  to 
maintain  specified  Standards  in  Training  Commission  (STRAC) 
readiness  conditions.  In  addition  to  justifying  the  ammuni¬ 
tion  allocations,  the  resulting  data  base  will  be  used  to 
develop  a  new  gunnei'y  training  manual. 

The  second  major  propiem  is  the  lack  of  adequace  ranges 
on  which  to  train  and  qualify  unit  aviators.  Many  of  the 
ranges  presently  available  lack  the  targetry,  scoring 
devices,  and  space  required  for  effective  training.  Further¬ 
more,  limited  access  to  the  ranges  inhibit  the  gunnery 
training  and  make  it  difficult  to  maintain  the  required 
readiness  conditions.  The  research  is  intended  to  document 
the  availability,  type,  and  utility  of  gunnery  ranges 
currently  in  use  bv  Army  aviation  units. 


The  final  major  problem  is  the  lack  of  empirical  data  on 
the  utility  of  flight  simulators  for  weapon  systems  training. 
Theoretically,  flight  simulators  can  reduce  the  impact  of  the 
first  two  problems.  That  is,  weapons  training  can  be  con¬ 
ducted  without  physical  ranges  and  without  incurring  ammuni¬ 
tion  costs.  However,  there  are  no  systematic  data  about  what 
should  be  trained  in  simulators,  the  amount  of  training  that 
is  most  cost-effective,  or  the  extent  that  flight  simulator 
training  may  offset  the  need  for  weapon  firing  in  the  air¬ 
craft.  This  problem  is  compounded  by  the  single  configura¬ 
tion  of  the  AH-1  attack  helicopter  flight  simulator  (AH1SFWS) 
that  is  used  by  unit  aviators  who  fly  different  configura¬ 
tions  of  AH-1  attack  helicopters  {e.g.,  AH-1G,  AH-1M,  AH-1S 
Modified,  AH-1S  Production,  and  AH-1S  ECAS) . 


In  January  1987,  the  Commanding  General  of  the  USAAVNC 
directed  that  a  survey  of  field  unit  aviators  and  aviation 
unit  commanders  be  conducted  to  compile  the  research  data 
required  to: 

•  formulate  an  accurate  ammunition  procurement  request, 

•  support  revisions  of  the  Army's  aerial  gunnery 
training  programs,  and 

•  evaluate  the  training  value  of  flight  simulators  for 
aerial  gunnery. 

The  survey  is  designed  to  be  administered  worldwide  to  all 
active  Army  and  National  Guard  attack  helicopter  aviators  and 
unit  commanders.  Despite  the  extensive  nature  of  the 
project,  reporting  commitments  dictate  that  the  survey  be 
developed  and  administered,  and  the  data  analyzed  by  December 
1987. 

The  research  activities  are  divided  into  three  phases. 
Phase  1  involves  the  development  of  the  A&G  survey  and  the 
required  ancillary  materials.  Phase  2  involves  the  adminis¬ 
tration  of  the  A&G  survey  to  the  unit  aviators  and  unit 
commanders.  Phase  3  involves  the  processing,  analysis,  and 
reporting  of  the  survey  data. 
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Project  Status 

Work  Completed 

Phase  1:  Survey  development.  Following  a  review  of  the 
relevant  literature,  the  current  aerial  gunnery  training 
manual  (Headquarters,  Department  of  the  Army  Field  Manual  FM 
1-140,  1986) ,  a  previous  STRAC  questionnaire,  and  the 
Essential  Elements  of  Analysis  delineated  by  the  DGFS  Study 
Group,  approximately  100  preliminary  survey  items  were 
drafted  that  covered  the  following  ten  topics: 

•  personal  data  about  the  respondent, 

•  military  experience  of  the  respondent, 

•  flight  experience  of  the  respondent, 

•  present  duty  assignment  of  the  respondent, 

•  suitability  of  current  gunnery  training  publications, 

•  weapon  systems  on  the  aviator's  primary  aircraft, 

•  ammunition  allocated  and  fired  during  the  1987 
training  year, 

•  utilization  of  gunnery  range  facilities, 

•  utility  of  flight  simulators  for  gunnery  training  and 
qualification,  and 

•  door  gunnery  training. 

The  preliminary  survey  items  were  administered  to 
approximately  50  attack  helicopter  aviators  by  DGFS  person¬ 
nel.  The  results  of  this  pretest  were  used  to  produce  a 
second  draft  of  the  A&G  survey.  The  second  draft  was  divided 
into  two  forms:  Form  A  for  the  unit  aviators  and  Form  B  for 
the  unit  commanders.  Many  of  the  items  on  the  two  forms  are 
similar  in  content,  but  the  unit  aviator  responds  to  the 
items  with  respect  to  himself  and  the  unit  commander  responds 
to  the  items  with  respect  to  the  entire  unit. 

The  second  draft  of  the  survey  was  not  pretested  because 
of  administrative  problems  at  th*3  participating  installa¬ 
tions.  DGFS  personnel,  acting  as  aviator  subject  matter 
experts  (SMEs),  and  Anacapa  Sciences,  Inc.,  personnel 
reviewed  and  edited  the  final  versions  of  the  survey  forms 
and  prepared  the  required  ancillary  materials  (e.g.,  letters 
of  instruction) .  The  surveys  were  subsequently  approved  for 
use  by  the  U.S.  Army  Soldier  Support  Center  and  then 
rep  o  iuced  for  administration  by  DGrS. 

Form  A  contains  68  items  divided  into  nine  of  the  ten 
topic  areas  listed  above;  no  questions  are  posed  to  the  unit 
aviators  about  door  gunnery.  Form  B  contains  78  items 
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divided  into  all  ten  topic  areas.  The  surveys  are  much  more 
comprehensive  than  the  number  of  items  indicates.  That  is, 
many  items  have  multiple  sections  or  require  a  succession  of 
responses.  Although  no  respondent  would  complete  all  the 
items,  there  are  472  codable  responses  on  Form  A  and  644 
codable  responses  on  Form  B.  In  addition,  both  forms  have 
several  open-ended  response  items  that  cannot  be  coded  for 
entry  into  the  computer  data  base. 

Phase  2:  Survey  data  collection.  DGFS  personnel  have 
distributed  the  two  survey  forms  and  ancillary  materials  to 
all  active  Army  and  National  Guard  aviation  units  that  have 
attack  helicopter  missions.  During  August  1987,  approxi¬ 
mately  300  commander  forms  and  2000  aviator  forms  were 
distributed  to  the  active  Army  and  National  Guard  units.  At 
the  end  of  September  1987,  approximately  80  unit  commander 
forms  and  400  unit  aviator  forms  had  been  completed  and 
returned  to  DGFS  for  processing. 

Phase  3; _ Data  processing  and  analysis .  AR I ARD A 

personnel  have  developed  computer  programs  to  enter  and 
verify  the  data  collected  by  the  A&G  survey.  The  survey 
responses  were  entered  and  verified  by  DGFS  personnel  as  the 
forms  were  returned  from  the  aviators  and  commanders . 

Several  meetings  were  held  with  DGFS  personnel  to 
enumerate  the  most  important  research  issues,  to  identify  the 
subset  of  survey  items  that  most  directly  address  those 
issues,  and  to  develop  a  statistical  approach  for  analyzing 
the  items.  Subsequently,  computer  programs  were  prepared  to 
analyze  approximately  one-third  of  the  items  on  each  form, 
and  a  preliminary  analysis  was  conducted  on  the  data  that  had 
been  entered  and  verified.  The  results  of  the  preliminary 
analyses  are  not  presented  in  this  report  because  the  current 
sample  is  not  representative  of  the  population  (i.e.,  most  of 
the  survey  responses  to  date  have  been  received  from  units  in 
the  Federal  Republic  of  Germany) . 


Work  Projected 

Phase  2:  Survey  data  collection.  Approximately  25%  of 
the  surveys  that  were  distributed  have  been  completed  and 
returned  for  processing.  Data  collection  efforts  will  con¬ 
tinue  until  the  end  of  October  1987.  In  particular,  efforts 
will  be  made  to  improve  the  representativeness  of  the  sample 
by  enccuraging  unit  aviators  and  commanders  in  the  United 
States  and  Korea  to  respond  to  the  surveys. 
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Phase  3:  Data  processing  and  analysis.  Data  processing 
will  continue  until  the  data  collection  effort  is  terminated. 
When  all  the  survey  responses  are  entered  and  verified,  the 
primary  data  will  be  analyzed  and  briefings  will  be  prepared 
for  conferences  that  are  scheduled  for  November  and  December 
1987.  Subsequently,  secondary  analyses  will  be  conducted  and 
a  final  report  will  be  prepared  in  collaboration  with  DGFS 
personnel . 


:opter  Gunnery  (FM  1-140)  (1986,  October).  Washington, 

DC:  Headquarters,  Department  of  the  Army. 


UTILIZATION/EFFECTIVENESS  OF  FLIGHT  SIMULATORS 
FOR  FIELD  UNIT  TRAINING 

Dr.  George  L.  Kaempf,  Project  Director 


Background 

The  Army's  Synthetic  Flight  Training  System  (SFTS)  has 
been  audited  by  the  Army  Audit  Agency  (AAA)  on  two  occasions, 
first  in  1981  and  again  in  1984.  The  results  of  the  first 
audit  are  described  in  AAA  Audit  Report  SO  82-6,  (U.S.  Army 

Audit  Agency,  1982);  the  results  of  the  second  are  described 
in  AAA  Audit  Report  SO  85-18  (U.S.  Army  Audit  Agency,  1985)  . 
The  primary  issue  in  both  audit  reports  is  the  number  of 
flight  simulators  required  to  support  the  training  of  field 
unit  aviators.  Specifically,  both  reports  concluded  that  the 
Army  had  not  adequately  quantified  the  return  on  its  invest¬ 
ment  in  flight  simulators  dedicated  to  field  unit  training. 
The  potential  returns  include  benefits  such  as  reductions  in 
the  number  of  aircraft  flight  hours  and  increases  in  the 
training  effectiveness  and  combat  readiness  level  of  the 
Army's  aviator  force. 

The  AAA  reports  also  admonished  the  Army  for  the  manner 
in  which  operational  tests  had  been  conducted  on  the  SFTS. 

The  reports  stated  that  (a)  the  operational  tests  should  have 
been  conducted  in  a  realistic  operational  environment  with 
unit  aviators,  (b)  the  Army  made  unwarranted  assumptions 
during  the  analyses  of  the  operational  test  data,  (c)  appro¬ 
priate  methods  and  controls  were  not  employed  during  opera¬ 
tional  testing,  and  (d)  certain  training  capabilities 
required  by  the  simulator  specifications  were  not  evaluated 
in  the  operational  tests  (e.g.  nap-of-the-earth  and  night 
flight) .  In  short,  the  AAA  concluded  that  adequate  opera¬ 
tional  tests  had  not  been  performed  to  verify  the  putative 
benefits  of  using  flight  simulators  as  part  of  continuation 
training.  Following  the  1981  audit,  AAA  concluded  that, 
although  flight  simulators  had  reduced  the  training  costs  and 
improved  training  at  the  U.S.  Army  Aviation  Center  (USAAVNC) , 
the  Army  had  not  determined  the  effects  that  employment  of 
the  flight  simulators  would  have  on  training  in  aviation 
field  units. 

The  purpose  of  the  follow-up  audit  in  1984  was  to 
determine  whether  appropriate  corrective  actions  had  been 
taken  in  response  to  the  recommendations  of  the  previous 
report.  During  the  second  audit,  AAA  recognized  that  the 
Army  had  changed  its  basis  for  justification  of  flight 
simulation.  Previously,  the  Army  had  based  its  development 
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of  the  SFTS  program  on  the  premise  that  simulators  pay  for 
themselves  by  reducing  flight  hour  costs.  Subsequently,  the 
Army  took  the  position  that  the  number  of  hours  flown  in  Army 
aircraft  had  been  reduced  to  the  absolute  minimum  and  that 
flight  simulators  should  be  justified  by  their  ability  to 
enhance  training  effectiveness  and  combat  readiness.  How¬ 
ever,  the  1985  audit  report  concluded  that  the  Army  had  not 
implemented  the  previous  recommendations  to  quantify  these 
benefits . 

The  overriding  issue  in  both  audit  reports  was  the 
number  of  flight  simulators  that  are  required  to  support  the 
training  of  field  unit  aviators;  specifically,  whether  unit 
training  requirements  can  be  met  with  fewer  flight  simulators 
than  are  specified  in  the  Army's  Basis  of  Issue  Plans 
(BOIPs) .  In  their  audit  reports,  the  AAA  strongly  emphasized 
that  both  the  BOIPs  and  the  AAA  analyses  of  flight  simulator 
requirements  are  based  on  vague  information  about  the  roles 
that  flight  simulators  are  to  play  in  field  unit  training. 

The  AAA  strongly  urged  the  Army  to  undertake  research  needed 
to  quantify  the  return  on  the  Army's  investment  in  flight 
simulators  that  are  to  be  employed  in  operational  field 
units . 

In  response  to  the  recommendations  from  AAA,  Anacapa 
Sciences,  Inc.,  researchers  have  accomplished  work  during  the 
first  contract  year  under  two  separate  but  related  taskings . 
The  first  tasking  was  received  from  the  U.S.  Army  Aviation 
Center's  (USAAVNC)  Directorate  of  Training  and  Doctrine 
(DOTD )  in  June,  1984.  The  second  tasking  was  received  from 
the  Department  of  the  Army  (DA)  in  June,  1986.  The  remainder 
of  this  section  presents  the  background  for  each  of  these 
taskings . 


DQID  Tasking 

The  DOTD  formally  tasked  the  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
(ARIARDA)  to  provide  research  support  that  would  address  the 
issues  raised  about  the  Army's  SFTS  training  program. 
Specifically,  DOTD  requested  that  ARIARDA  conduct  research  to 
answer  such  questions  as: 

•  What  tasks  can  best  be  trained  in  the  flight 
simulators? 

•  What  rate  of  practice  in  the  flight  simulators  best 
enables  aviators  to  maintain  proficiency? 

•  How  can  the  flight  simulators  be  used  to  maintain 
proficiency  on  cognitive  skills  that  are  not  routinely 
used  in  the  aircraft? 
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•  What  impact  does  ammunition  reduction  have  on  training 
programs,  aviator  proficiency,  and  unit  readiness? 

•  How  can  the  flight  simulators  be  used  to  provide  night 
vision  goggle  (NVG)  training? 

Furthermore,  it  is  generally  recognized  that  the 
following  five  factors  must  be  considered  in  assessing  the 
return  on  the  investment  in  flight  simulators: 

•  the  cost  of  acquiring,  housing,  operating,  and 
maintaining  the  flight  simulators; 

•  the  cost  of  transporting  unit  aviators  to  the  flight 
simulators ; 

•  the  number  of  aviators  to  be  trained  in  the  flight 
simulators ; 

•  the  amount  of  flight  simulator  training  each  aviator 
will  receive;  and 

•  the  benefits  of  the  flight  simulator  training. 

Information  on  the  first  three  factors  is  available  or  can 
easily  be  obtained.  However,  little  information  is  available 
on  the  last  two  factors.  The  current  research  is  designed  to 
determine  empirically  the  type  and  amount  of  training  unit 
aviators  should  receive  in  flight  simulators,  and,  to  the 
extent  possible,  quantify  the  benefits  of  this  training. 


Early  in  1986,  the  Office  of  the  Deputy  Chief  of  Staff, 
Operations  (ODCSOPS)  reviewed  the  issues  concerning  the 
development  of  training  programs  that  utilize  flight  simula¬ 
tion  and  the  fielding  of  the  Ar ay ' s  flight  simulators.  DA 
determined  that  training  effectiveness  analyses  should  be 
conducted  for  each  of  the  Army's  flight  simulation  systems  to 
serve  as  the  basis  for  the  development  of  effective  training 
strategies.  Accordingly,  DA  tasked  the  Training  and  Doctrine 
Command  (TRADOC)  to  develop  and  implement,  with  the  assis¬ 
tance  of  ARIARDA,  post-fielding  training  effectiveness 
analyses  (PFTEA)  of  the  AH-1,  AH-64,  UH-60,  and  the  CH-47 
flight  simulator  systems. 

Representatives  from  ARIARDA,  Anacapa,  DOTD ,  and  the 
TRADOC  Analysis  Center  at  White  Sands  Missile  Range 
(TRACWSMR)  met  on  1  July  1986  to  review  the  tasking  and 
identify  a  course  of  action.  The  group  members  found  that 
the  issues  involved  in  determining  the  effectiveness  of 
simulators  for  training  aviators  at  the  USAAVNC  were 
substantially  different  from  the  issues  involved  in  deter¬ 
mining  the  effectiveness  of  simulators  for  training  aviators 
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in  operational  field  units.  Therefore,  the  group  members 
agreed  that  TRACWSMR  would  develop  and  implement  a  research 
plan  to  address  the  issues  of  employing  simulation  within  the 
institutional  environment  and  that  ARIARDA  would  develop  and 
implement  a  plan  to  address  the  utilization  of  flight 
simulators  in  field  environments. 


The  Army  has  purchased  and  installed  five  AH-1  Flight 
and  Weapons  Simulators  (FWSsj  at  Forces  Command  (FORSCOM)  and 
U.S.  Army,  Europe  (USAREUR)  sites;  three  additional  AHlFWSs 
are  scheduled  for  installation  prior  to  1990.  In  addition, 
the  Army  plans  to  deploy  r even  AH-64  Combat  Mission 
Simulators  (CMSs),  six  CH-47D  Flight  Simulators  (CH47FSs), 
and  sixteen  UH-60  Flight  Simulators  (UH60FSs)  to  various 
field  locations.  With  the  exception  of  the  few  flight 
simulators  located  at  Fort  Rucker,  tne  Army  has  earmarked  its 
flight  simulators  for  use  by  operational  field  units. 

The  Army  is  deploying  its  flight  simulators  with  the 
primary  intention  cf  providing  operational  field  units  with 
training  devices  in  which  aviators  may  sustain  their  flight 
and  tactical  skills.  However,  very  little  empirical  data 
currently  exist  (a)  to  support  the  notion  that  flight 
simulators  effectively  and  efficiently  provide  this  type  of 
training,  and  (b)  to  guide  the  Army  in  developing  training 
programs  that  include  a  satisfactory  mix  of  training 
conducted  in  the  aircraft  and  flight  simulator. 


Project  Objectives 

The  objectives  of  the  research  resulting  from  the  DOTD 
and  the  DA  taskings  are  to: 

•  identify  those  tasks  that  can  be  sustained/maintained 
effectively  and  efficiently  in  the  AH1FWS; 

•  provide  data  of  sufficient  quality  to  serve  as  a  basis 
for  the  development  of  training  strategies  that  incor¬ 
porate  a  sound  mix  of  flight  simulator  and  aircraft 
flight  training  within  an  operational  field  unit 
environment;  and 

•  identify,  to  the  extent  possible,  training  techniques 

and  strategies  appropriate  for  aviators  to  maintain 
their  flight  and  tactical  skills  in  flight  simulators.  V.j 


In  response  to  the  DOTD  tasking,  ARIARDA  assembled  a 
team  consisting  of  research  psychologists,  aviators,  and 
simulation  experts  to  address  the  research  questions.  The 
team  developed  a  research  plan  designed  to  answer  the  issues 
raised  by  the  first  AAA  audit  as  well  as  additional  issues 
identified  by  other  simulation  experts.  This  plan  is 
presented  in  detail  by  Cross  et  al  (1984)  and  Cross  and 
Gainer  (1985)  .  The  research  plan  calls  for  a  series  of 
studies  at  the  USAAVNC  and  in  operational  field  units.  The 
results  of  some  of  these  studies  have  been  reported 
previously  (see  Kaempf,  1986)  .  The  current  study  in  the 
series  is  a  transfer  of  training  study  in  the  AH1FWS  for 
emergency  touchdown  maneuvers  (ETMs) . 


DA  Taskinc 


In  support  of  the  DA  tasking,  Anacapa  researchers 
developed  a  plan  that  addresses  the  issues  identified  by  DA 
and  submitted  the  document  to  DOTD,  TRADOC,  and  DA  for 
approval.  The  plan  proposes  a  single  research  strategy  that 
can  be  applied  to  each  of  the  four  simulator  systems  with 
minor  modifications  to  provide  for  the  unique  characteristics 
of  each  system.  The  PFTEA  overview  section  below  briefly 
discusses  the  proposed  approach. 


PFTEA  overview.  The  PFTEA  is  intended  to  generate 
empirical  data  needed  to  make  decisions  about  the  employment 
of  Army  flight  simulators  to  train  and  sustain  the  flying 
skills  of  field  unit  aviators.  The  program  will  generate  the 
data  to  support  management  decisions  on  the  optimal  mix  of 
aircraft  and  simulator  training  for  skill  sustainment 
including,  where  appropriate,  the  substitution  of  simulator 
hours  for  aircraft  hours. 


The  PFTEA  proposes  to  investigate  the  effectiveness  of 
each  simulator  system  to  sustain  aviator  flight  skills  on 
groups  of  selected  flight  tasks.  Several  groups  of  flight 
tasks  will  be  investigated  for  each  different  simulator 
system;  each  task  group  investigated  will  require  a  separate 
sample  of  subjects.  The  task  groups  include: 

»  emergency  touchdown  maneuvers, 

•  basic  flight  tasks, 

•  flight  with  night  vision  systems,  and 

•  tactical  and  weapons  tasks. 


The  PFTEA  calls  for  a  one-year  study  of  each  task  group 
for  each  simulator  system.  Aircraft  checkrides  on  the  tasks 
of  interest  will  be  given  at  the  beginning  of  the  year.  A 
control  group  of  ten  aviators  will  continue  their  normal 
training  routine  in  the  aircraft  and  will  not  practice  in  the 
flight  simulator.  Three  experimental  groups  of  ten  aviators 
each  will  practice  the  same  tasks  under  controlled  training 
conditions  in  the  simulator  for  a  period  of  one  year.  The 
three  experimental  groups  will  differ  only  in  the  frequency 
of  simulator  training.  That  is,  they  will  receive  training 
to  proficiency  in  the  flight  simulator  either  monthly, 
quarterly,  or  semiannually.  Except  for  the  limited  group  of 
tasks  under  investigation,  the  experimental  groups  will 
continue  with  their  normal  aircraft  flying  routine  throughout 
the  period  of  the  study.  At  the  end  of  the  year,  each 
subject  will  be  administered  another  aircraft  checkride  on 
the  tasks  of  interest. 

The  sustainment  effectiveness  of  the  simulator  will  be 
determined  by  comparing  the  aircraft  checkride  performance  of 
the  experimental  groups  with  that  of  the  control  group. 
Aviators  in  any  of  the  four  groups  found  to  perform  below 
standards  on  these  tasks  during  the  end-of-year  checkride 
will  receive  additional  aircraft  training  until  they  reach 
proficiency,  and  the  amount  of  such  supplemental  training 
will  be  measured. 

In  addition,  the  training  effectiveness  of  the  flight 
simulators  will  be  investigated  by  using  groups  of  aviators 
who  are  completely  restricted  from  aircraft  flying  for  a 
period  of  one  year.  For  each  aircraft  system,  sixty  rated 
and  current  Flight  Activity  Category  (FAC)  2  aviators  will  be 
selected  to  serve  as  subjects.  The  subjects  will  be  given  a 
pretest  checkride  in  the  aircraft  on  all  FAC  2  Aircrew 
Training  Manual  (ATM)  tasks  and  then  assigned  to  groups  in  a 
manner  that  produces  four  groups  (three  experimental  groups 
of  10  aviators  each  and  one  control  group  of  30  aviators) 
matched  on  their  baseline  levels  of  performance  and  flight 
experience.  After  the  pretest  checkride,  the  control  group 
will  be  prohibited  from  all  flying  for  a  period  of  one  year, 
and  then  administered  another  aircraft  checkride  at  the  end 
of  that  period. 

Following  the  pretest  checkride,  the  experimental  groups 
will  begin  a  one  year  training  program  that  consists  only  of 
periodic  training  to  proficiency  in  the  flight  simulator. 

The  subjects  will  be  prohibited  from  flying  in  the  aircraft. 
The  three  experimental  groups  will  differ  only  in  the  fre¬ 
quency  of  simulator  training.  That  is,  they  will  receive 
training  to  proficiency  in  the  flight  simulator  either 
monthly,  quarterly,  or  semiannually.  At  the  end  of  the 


one-year  period,  the  subjects  in  the  experimental  groups  will 
be  administered  another  checkride  in  the  aircraft.  All  skill 
deficiencies  identified  during  the  second  checkride  in  either 
group  will  be  corrected  through  additional  training  in  the 
aircraft . 


Project  Status 

DOTD  Tasking:  Work  Completed 

During  coordination  to  secure  resources  for  the  AH-1 
transfer  of  training  study  of  emergency  touchdown  maneuvers, 
FORSCOM  stated  that  the  study  required  more  resources  than 
any  one  location  or  unit  could  provide.  At  FORSCOM' s 
request,  ARIARDA  conducted  the  study  at  two  locations:  Fort 
Campbell,  Kentucky,  and  Fort  Lewis,  Washington.  Therefore, 
the  researchers  first  collected  data  at  Fort  Campbell  on  half 
of  the  subjects  (five  control  group  subjects  and  five  experi¬ 
mental  group  subjects)  and  then  replicated  the  procedures 
with  another  ten  subjects  at  Fort  Lewis. 

Maneuvers .  The  researchers  identified  five  emergency 
touchdown  maneuvers  for  investigation.  Full  descriptions  of 
each  maneuver  and  their  performance  standards  can  be  found  in 
the  AH-1  ATM  (Department  of  the  Army,  1984) .  The  five 
maneuvers  were: 

•  Standard  Autorotation  (ATM  Task  3001) , 

•  Low-Level  Autorotation  (ATM  Task  3002) , 

•  Low-Level  High-Speed  Autorotation  (ATM  Task  3005) , 

•  Simulated  Right  Antitorque  Failure  (ATM  Task  3004), 
and 

•  Simulated  Dual  Hydraulics  Failure  (ATM  Task  3003) . 

Subjects .  Twenty  qualified  and  current  AH-1  pilots 
served  as  subjects  for  the  study.  Demographic  information 
was  collected  for  the  control  and  experimental  group 
subjects.  No  significant  differences  existed  between  the  two 
groups  on  any  of  the  demographic  variables  reported. 

Evaluators .  Three  current  and  highly  experienced  AH-1 
instructor  pilots  (IPs)  performed  all  of  the  evaluations  and 
training  conducted  in  the  aircraft  at  Fort  Campbell  and  Fort 
Lewis.  Six  trained  evaluators  conducted  all  of  the  check- 
rides  and  training  in  the  AH1FWS  at  both  sites.  All  of  the 
evaluators  and  IPs  were  provided  a  minimum  of  ten  hours  of 
training  to  learn  how  to  use  the  gradeslips  and  evaluation 
procedures  employed  in  this  study. 
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Performance  measures .  The  evaluators  completed  a  two 
part  gradeslip  for  each  trial  attempted  during  checkrides  and 
training.  A  series  of  descriptive  rating  scales  compose  the 
first  portion  of  each  gradeslip.  The  descriptive  scales  are 
13-point  bidirectional  scales  designed  to  indicate  the 
magnitude  and  direction  of  deviations  from  ideal  performance 
on  a  number  of  parameters  relevant  to  each  maneuver.  The 
scales  are  anchored  to  established  standards  and  performance 
tolerances  for  each  parameter,  The  evaluators  completed  the 
descriptive  scales  as  the  trial  was  in  progress.  The 
descriptive  scales  serve  three  purposes.  First,  they  iden¬ 
tify  the  aspects  of  each  maneuver  that  are  most  difficult  for 
the  subjects.  Second,  the  scales  serve  as  guides  for  the 
evaluators  to  rate  the  overall  performance  for  each  maneuver. 
Finally,  the  scales  provide  information  about  the  criteria 
the  evaluators  employ  when  assessing  overall  aviator 
performance . 

On  the  second  portion  of  each  gradeslip,  the  IPs 
provided  subjective  evaluations  of  overall  pilot  performance 
that  were  the  principal  performance  measures  for  this  study. 
Observers  evaluated  all  trials  attempted  by  each  subject  in 
the  aircraft  and  the  AH1FWS.  The  evaluators  derived  subjec¬ 
tive  ratings  of  overall  performance  on  each  trial  from  a  15- 
point  scale.  Six  values  (1-6)  on  the  scale  represent 
unsatisfactory  performance  and  nine  values  (7-15)  represent 
varying  degrees  of  satisfactory  performance.  Immediately 
following  completion  of  a  trial,  the  evaluator  subjectively 
determined  whether  performance  had  been  satisfactory  or 
unsatisfactory,  and  then  assigned  a  value  within  the  range  of 
the  subscale. 

Procedures .  As  indicated  previously,  Anacapa  conducted 
the  study  at  two  sites.  Subjects  always  flew  the  maneuvers 
from  the  pilot's  station  in  both  the  aircraft  and  the  AH1FWS. 
Unless  otherwise  noted  in  this  section,  the  researchers 
followed  identical  procedures  at  both  sites.  The  researchers 
collected  data  for  ten  days  at  each  site  in  the  following 
steps  : 

•  Prior  to  the  beginning  of  the  study,  the  researchers 
briefed  all  participants  about  the  purpose  and  proce¬ 
dures  of  the  study.  Then,  the  subjects  completed  a 
written  test  of  their  knowledge  of  the  procedures  and 
standards  for  the  five  maneuvers  being  investigated. 

•  Each  subject  completed  an  initial  checkride  in  the 
aircraft  on  Day  1  of  the  study.  The  checkride 
consisted  of  one  trial  of  each  of  the  five  maneuvers, 
with  the  order  of  maneuvers  counterbalanced  across 
subjects  . 
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•  Following  the  aircraft  checkride,  each  subject  was 
administered  an  initial  checkride  in  the  AH1FWS  on 
Day  1.  The  simulator  checkride  was  identical  to  the 
aircraft  checkride  in  terms  of  content  and  order  of 
maneuvers . 

•  The  researchers  assigned  subjects  to  control  and 
experimental  groups  (five  subjects  per  group  at  each 
site)  based  on  the  overall  performance  ratings  awarded 
during  the  initial  aircraft  checkrides.  The  assign¬ 
ment  of  subjects  to  groups  equated  the  two  groups  on 
the  baseline  level  of  performance  in  the  aircraft  for 
the  five  maneuvers. 

•  The  IPs  trained  each  of  the  subjects  in  the  control 
group  to  proficiency  on  all  five  maneuvers  in  the 
aircraft,  beginning  on  Day  2. 

•  The  evaluators  trained  each  subject  in  the  experi¬ 
mental  group  to  proficiency  on  all  five  maneuvers  in 
the  AH1FWS,  beginning  on  Day  2. 

•  Following  training  in  either  the  aircraft  or  the 
AH1FWS,  each  subject  received  a  second  checkride  in 
the  AH1FWS .  The  second  checkride  in  the  AH1FWS  was 
identical  to  the  subject's  first  simulator  checkride, 
and  it  occurred  no  later  than  three  days  following 
completion  of  initial  training. 

•  The  IPs  trained  each  of  the  experimental  group 
subjects  to  proficiency  on  all  five  maneuvers  in  the 
aircraft.  This  training  began  no  later  than  the  day 
following  completion  of  the  subject's  second  checkride 
in  the  AH1FW3 . 

The  rating  procedures  followed  in  the  aircraft  and  the 
AH1FWS  were  not  identical.  Due  to  space  limitations,  only 
one  evaluator  could  observe,  evaluate,  and  instruct  aviator 
performance  on  any  given  trial  in  the  aircraft.  The  lone 
evaluator  in  the  aircraft  occupied  the  copilot/gunner  (CPG) 
station  and  completed  the  descriptive  scales  of  the  gradeslip 
during  and  immediately  following  each  trial. 

Two  evaluators  observed  and  evaluated  all  trials  com¬ 
pleted  in  the  AH1FWS.  One  evaluator  occupied  the  operator's 
console  at  the  pilot's  station  to  operate  tne  simulator  and 
to  evaluate  the  subjects'  performance.  The  other  evaluator 
occupied  the  simulator's  CPG  station  to  monitor  the  flight 
controls  and  other  parameters.  Both  evaluators  completed 
separate  gradeslips  for  each  trial.  Following  completion  of 
each  trial,  they  collaborated  to  derive  one  gradeslip  that 
was  used  for  data  analysis.  This  collaboration  included 
discussions  between  the  evaluators,  references  to  the  grade- 
slips  they  had  completed  individually,  and  observations  of 


replays  of  the  trial  to  ensure  that  the  gradeslips  were 
completed  as  accurately  as  possible.  All  data  collected 
during  the  transfer  of  training  study  of  ETMs  were  entered 
into  a  computer  data  base  and  verified  for  accuracy. 

During  the  training  portions  of  the  study,  the  evalua¬ 
tors  trained  the  subjects  to  proficiency  on  each  of  the  five 
maneuvers.  No  effort  was  made  to  control  the  order  in  which 
the  maneuvers  were  trained.  The  researchers  defined  profi¬ 
ciency  as  two  consecutive  trials  of  a  given  maneuver  on  which 
the  subject  received  a  satisfactory  overall  performance 
rating.  As  a  subject  reached  proficiency  on  a  maneuver,  that 
maneuver  was  dropped  while  training  continued  on  the 
remaining  maneuvers.  During  their  last  training  sessions, 
the  subjects  may  have  practiced  only  one  or  two  maneuvers. 


Analyses  of  the  ETM  data  will  be  conducted  and  reported. 
In  addition,  the  results  of  the  ETM  study  and  ARIARDA's 
recommendations  will  be  briefed  to  interested  Army  agencies 
at  USAAVNC,  Fort  Campbell,  and  Fort  Lewis. 

Anacapa  will  continue  to  address  issues  specified  by  the 
tasking  received  from  DOTD  in  1984.  Accordingly,  ARIARDA 
submitted  requests  for  FORSCOM  support  for  four  research 
projects  during  the  fiscal  year  1988.  Pending  FORSCOM 
approval,  these  projects  will  be  conducted  during  the  next 
contract  year.  The  four  proposed  projects  are: 

•  observe  training  sessions  conducted  in  the  AH1FWS  to 
identify  simulator  specific  training  strategies 
currently  employed  by  AH-1  field  units; 

•  observe  AH-1  field  units  during  field  exercises  to 
determine  the  coordination  and  skills  required  for 
attack  crews  and  teams  to  operate  in  a  combat 
environment ; 

•  conduct  structured  interviews  with  AH-64  IPs  and 
commanders  to  identify  field  units'  training 
requirements  and  the  techniques  currently  employed  to 
train  AH-64  aviators;  and 

•  conduct  a  transfer  of  training  study  in  the  AH1FWS  and 
AH-1  aircraft  for  gunnery  tasks. 
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ARIARDA  submitted  the  PFTEA  research  plan,  Flight 
Simulator  Skill  Sustainment  Training  Effectiveness  Research 
Plan  (ARIARDA,  1986),  to  DOTD,  TRADOC,  and  DA  for  approval. 
DOTD  and  TRADOC  approved  the  plan  with  relatively  minor 
changes,  and,  in  June  1987,  DA  approved  the  PFTEA  research 
plan  . 

Due  to  the  projected  distribution  of  aviators  in  the 
various  Army  commands  and  the  simulator  fielding  schedules, 

DA  determined  that  the  AH-1  portion  of  the  PFTEA  should  be 
implemented  first  and  that  the  remaining  studies  should  be 
conducted  as  the  aviator,  aircraft,  and  simulator  resources 
become  available.  Anacapa  prepared  and  submitted  to  TRADOC  a 
list  of  the  resources  required  to  implement  the  AH-1  portion 
of  the  PFTEA  and  a  table  of  milestones  for  the  entire  project 
(ARIARDA,  1987) .  ARIARDA  predicted  that  the  AH-1  PFTEA  would 
require  28  months  and  the  entire  PFTEA  would  require  62 
months  to  complete.  At  the  end  of  the  first  contract  year, 
neither  TRADOC  nor  DA  had  responded  to  the  list  of  resource 
requirements  and  milestones. 

DA  sent  to  USAREUR  an  information  copy  of  the  original 
PFTEA  tasking.  USAREUR  responded  with  a  message  dated  31 
July  1986  expressing  its  support  for  the  proposed  project. 
ARIARDA  researchers  acknowledged  USAREUR 's  support  by  message 
and  travelled  to  Germany  to  brief  USAREUR  officials  on  the 
PFTEA  plan.  Representatives  of  the  USAREUR  Aviation  Safety 
and  Standardization  Board  and  the  7th  Army  Training  Command 
(ATC)  indicated  strong  interest  in  the  gunnery  segments  of 
the  PFTEA  and  a  willingness  to  coordinate  the  resources 
required  to  conduct  the  study  of  AH-1  gunnery  tasks.  USAREUR 
is  particularly  anxious  (a)  to  evaluate  the  effectiveness  of 
the  AH1FWS  for  sustaining  gunnery  skills,  (b)  to  establish  an 
optimal  mix  of  simulator  and  aircraft  training  for  gunnery 
tasks,  and  (c)  to  determine  the  number  of  rounds  of  ammuni¬ 
tion  required  for  aviators  to  maintain  their  proficiency  in 
the  aircraft . 

Subsequently,  the  7th  ATC  began  coordinating  the 
resources  required  to  conduct  the  PFTEA  in  USAREUR,  but  two 
obstacles  remain.  First,  the  7th  ATC  must  staff  the  entire 
project  by  identifying  and  scheduling  specific  units  to  par¬ 
ticipate.  At  the  end  of  the  first  contract  year,  the  7th  ATC 
is  working  on  the  coordination  task,  with  a  suspense  date  of 
13  November  1987.  Second,  the  Directorate  of  Gunnery  and 
Flight  Systems  (DGFS)  at  the  USAAVNC  must  develop  new  pre¬ 
scriptive  gunnery  qualification  tables  for  the  AH-1.  These 
tables  will  replace  the  tables  in  the  manual  Helicopter 
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Gunnery  (Department  of  the  Army,  1986)  and  will  serve  as  the 
bases  for  performance  evaluation  and  training  in  the  PFTEA. 
DGFS  has  agreed  to  provide  draft  tables  no  later  than  16 
November  1987. 

i 

i 

i 

Work  Projected 

Planning  and  coordination  for  the  PFTEA  will  continue. 

1  Pending  completion  of  the  support  requirements  being  coordi- 

!  nated  by  the  7th  ATC,  Anacapa  tentatively  plans  to  begin  data 

collection  in  Germany  for  the  AH-1  gunnery  PFTEA  during 
i  February,  1988. 
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DEVELOPMENT  OF  AN  UPGRADED  BASIC  MITAC 
USING  AN  INTERACTIVE  VIDEODISC  FORMAT 

Mr.  Claude  0.  Miles,  Project  Director 


Background 

During  nap-of-the-earth  (NOE)  flight,  an  aviator  flies 
at  varying  speeds  as  close  as  possible  to  the  earth's 
surface — preferably  flying  around  obstacles  instead  of  over 
them — to  escape  radar  or  visual  detection  by  a  potential 
enemy.  Among  the  factors  that  make  NOE  navigation  different 
from  and  more  difficult  than  navigating  at  higher  altitudes 
are : 

•  a  limited  viewing  distance  when  operating  in  close 
proximity  to  the  ground, 

•  the  perspective  at  which  checkpoint  features  are 
viewed, 

•  the  difference  in  perspective  between  the  map 
representation  and  the  point  of  regard  of  the  NOE 
navigator, 

•  the  need  for  more  precise  navigation  and  map 
interpretation,  and 

•  the  need  for  rapid  association  of  map  features  with 
their  real-world  counterparts. 

Previous  studies  indicate  that  most  difficulties  faced 
by  Army  aviators  trained  in  NOE  navigation  are  due  to  a  lack 
of  map  interpretation  and  terrain  analysis  skills.  At  the 
low  altitudes  of  terrain  flight,  map  interpretation  and 
terrain  analysis  skills  are  critical  for  maintaining 
orientation  (Holman,  1978). 

Therefore,  specialized  training  is  needed  to  ensure 
proficiency  and  mission  success  in  NOE  navigation  (Fineberg, 
Meister,  &  Farrell,  1978)  .  In  response  to  this  need,  the 
U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  ( ARIARDA)  developed  and  implemented  a 
system  for  training  Army  aviators  in  the  critical  skills 
required  to  navigate  successfully  and  to  maintain  NOE 
orientation  in  a  high  threat  environment .  This  training 
course,  referred  to  as  the  Map  Interpretation  and  Terrain 
Analysis  Course  (MITAC) ,  was  designed  to  teach  students  to 
interpret  1 : 50, 000-scale  topographic  maps  and  to  use  this 
knowledge  to  maintain  accurate  geographic  orientation  when 
flying  at  NOE  altitudes  (Bickley,  1978) .  Specifically,  MITAC 
teaches  students  the  cartographic  principles  governing  map 


compilation,  the  symbols  used  to  portray  features  on  the  map, 
and  the  methods  used  to  associate  topographic  features  with 
their  counterparts  on  the  map.  Cinematic  exercises  are  used 
to  train  students  to  navigate  at  NOE  altitude,  taking  advan¬ 
tage  of  concealment  or  "masking"  of  the  aircraft  afforded  by 
vegetation  and  terrain. 

The  MITAC  program  is  presented  in  13  instructional 
units.  The  course  begins  with  an  introductory  lesson,  in 
which  students  are  taught  the  basic  principles  of  cartography 
and  map  reading,  and  continues  through  more  complex  exercises 
that  improve  the  students'  NOE  navigation  skills.  A  series 
of  color  35-mm  slides  is  provided  to  teach  the  conventions 
and  selection  criteria  that  cartographers  use  in  creating 
1 : 50 , 000-scale  topographic  maps,  as  well  as  fundamental  map 
reading.  A  special  text  entitled  "Map  Interpretation  in  NOE 
Flight"  is  provided  to  supplement  the  course  material.  The 
features  covered  by  the  slides  and  text  include: 

•  hydrography 

•  vegetation, 

•  transportation  lines, 

•  buildings, 

•  miscellaneous  cultural  features,  and 

•  terrain  relief. 

A  narration  recorded  on  cassette  tapes  is  provided  to 
supplement  the  slides.  In  this  portion  of  the  program,  the 
student  is  taught : 

•  the  basis  for  the  classification  of  roads, 

•  the  coding  criteria  for  vegetation, 

•  the  methods  and  rules  employed  in  delineating  relief 
and  drainage, 

•  the  conventions  used  when  portraying  cultural 
features,  and 

•  generalization  and  displacement  practices  in 
cartographic  drafting  (Cross  &  Rugge,  1980)  . 

Additional  performance  oriented  exercises  that  emphasize 
analysis  of  terrain  features  and  their  representation  on  the 
map  are  presented  to  the  students  on  film.  The  students 
conduct  NOE  mission  planning,  identify  checkpoints,  and 
assess  terrain  masking.  The  filmed  scenarios  enable  the 
students  to  apply  the  principles  they  are  being  taught.  The 
exercises  begin  with  a  preflight  briefing;  students  then 
perform  navigation  training  exercises.  The  navigation 
training  exercises  require  students  to  maintain  orientation 
and  mark  checkpoints  on  a  map  while  viewing  a  motion  picture 
film  of  routes  flown  at  NOE.  The  units  include: 


•  a  contour  analysis  exercise, 

•  preflight  terrain  exercises, 

•  along-track  orientation  exercises, 

•  cross-track  orientation  exercises,  and 

•  corridor  orientation  exercises. 

All  navigation  exercises  are  followed  by  a  postflight 
debriefing  that  requires  students  to  view  the  filmed  route  a 
second  time  while  listening  to  a  prerecorded  commentary  that 
describes  the  topographic  features  along  the  filmed  route 
that  are  most  useful  for  maintaining  geographic  orientation. 


MITAC  is  presently  being  utilized  at  the  U.S.  Army 
Aviation  Center  (USAAVNC)  for  NOE  training  of  initial  entry 
rotary  wing  (IERW)  students  and  some  unit  aviators.  MITAC 
was  evaluated  at  USAAVNC  and  found  to  be  effective  in 
teaching  the  skills  required  for  NOE  navigation  (Holman, 

1978) .  Holman's  study  revealed  that  a  group  of  IERW  students 
trained  with  MITAC  navigated  NOE  routes  with  twice  the  speed 
and  one-third  the  errors  when  compared  with  an  equivalent 
group  of  IERW  students  not  trained  with  MITAC. 


Project  Objectives 

In  September  1983,  ARIARDA  requested  that  the  basic 
MITAC  course  be  revised  and  converted  to  an  interactive 
videodisc  format.  The  objectives  of  this  effort  are  to 
expand  and  improve  the  quality  of  the  original  course  and  to 
provide  a  more  effective  and  sophisticated  medium  for 
presenting  the  training.  Since  videodisc  technology  affords 
many  capabilities  not  available  with  the  presently  used 
audio/visual  equipment,  a  decision  was  made  to  convert  the 
entire  course  to  the  videodisc  format. 


Project  Status 

Work  Completed 

The  MITAC  Illustrated  Lectures  Manual  for  Infantrymen 
was  used  as  a  guide  in  developing  a  comprehensive  series  of 
illustrated  lectures  for  Basic  MITAC.  The  information 
contained  in  the  manual  has  been  rewritten,  making  it  appli¬ 
cable  to  Army  aviator  training.  The  lesson  material  has 
undergone  several  revisions  and  is  now  completed.  Following 
the  revision  of  the  illustrated  lecture  series,  an  exhaustive 
list  of  slides  needed  to  supplement  the  lectures  was  com¬ 
piled.  Some  slides  in  stock  were  used;  however,  it  was 
necessary  to  conduct  several  photographic  missions  to  obtain 
many  new  slides.  New  maps  were  obtained  and  additional 


slides  were  produced  to  illustrate  the  different  terrain 
features  on  the  new  maps.  All  of  the  usable  slides  have  been 
coordinated  with  the  script. 


Storyboards  have  been  written  to  specify  the  key 
technical  information  necessary  to  develop  the  videodisc 
products.  Thus,  the  Basic  MITAC  course  materials  have  been 
upgraded  and  the  production  phase  of  this  project  is  ready  to 
begin . 


Work  Projected 

The  following  tasks  must  be  performed  during  the  final 
phase  of  production: 

•  transfer  slides  to  tape, 

•  transfer  slides  and  maps  to  a  digital  data  base, 

•  generate  text  and  computer  graphics, 

•  provide  special  effects  (e.g.,  highlight  features  and 
create  graphic  overlays)  , 

•  narrate  scripts, 

•  perform  off-line  edit  of  videotapes,  and 

•  perform  on-line  final  edit  of  videotapes. 

Some  of  the  production  tasks  will  be  performed  in-house  and 
some  will  be  performed  under  contract  by  a  production  studio. 
The  specific  tasks  to  be  performed  in-house  will  be  deter¬ 
mined  and  the  needed  equipment  to  support  their  production 
will  be  obtained.  The  remaining  tasks  will  be  conducted 
under  contract  by  a  production  studio. 

During  the  post-production  phase,  all  materials  will  be 
transferred  to  a  one-inch  master  videotape  format.  The 
videotapes  will  be  edited  for  adherence  to  specifications. 

The  edited  videotapes  will  be  converted  to  a  videodisc 
format,  and  videodisc  masters  will  be  produced.  Replication 
videodiscs  will  be  pressed  from  the  master  videodiscs  in 
sufficient  quantities  to  support  Army  training  needs. 

Computer  programs  will  be  written  so  that  the  videodisc 
format  can  be  used  in  an  interactive  level  III  training 
system  comprised  of  a  microcomputer,  videodisc  player,  and 
monitor.  The  completed  interactive  videodisc  MITAC  training 
system  will  then  be  ready  for  training  effectiveness 
evaluation  prior  to  implementation  to  support  Army  aviation 
t  raining . 
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DEVELOPMENT  AND  EVALUATION  OF  THE  AH- 64 
SYMBOLOGY  TRAINING  PROGRAM 

Dr.  John  W.  Ruffner,  Project  Director 


Background 

The  AH-64  attack  helicopter  is  a  two-crewmember  aircraft 
designed  to  fly  nap-of -the-earth  (NOE)  missions,  day  or  night 
in  all  weather  conditions,  and  to  detect,  engage,  and  destroy 
enemy  armor.  To  provide  this  capability,  the  AH-64  is 
equipped  with  several  complex  flight  and  weapons  delivery 
systems.  Successful  operation  of  some  of  these  systems 
requires  that  the  pilot  and  copilot/gunner  (CPG)  be  able  to 
observe,  interpret,  and  integrate  both  visual  imagery  and 
symbolic  information  presented  on  visual  displays. 

The  AH-64  visual  display  systems  that  provide  informa¬ 
tion  to  the  pilot  and/or  the  CPG  are  the  Target  Acquisition 
and  Detection  System  (TADS) ,  the  Pilot  Night  Vision  System 
(PN VS),  and  the  Fire  Control  Symbol  Generator.  The  TADS  is 
used  by  the  CPG  for  target  search,  detection,  recognition, 
and  designation.  Three  sensors--the  forward-looking  infrared 
(FLIR) ,  the  day  television  viewing  system  (DTV) ,  and  direct 
view  optics  (DVO) --provide  the  CPG  with  visual  information 
to  detect  and  engage  targets  at  standoff  ranges  during  the 
day  or  night  and  in  adverse  weather  conditions.  The  PNVS 
orovides  FLIR  imagery  that  enables  the  pilot  to  fly  the  air¬ 
craft  at  night  and  during  degraded  visibility  conditions. 

The  fire  control  symbol  generator  superinrooses  flight  and 
weapons  symbology  on  the  imagery  displayed  by  the  TADS  and 
PNVS  . 
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Visual  imagery  and  symbology  from  the  TADS  and  the  PNVS 
can  be  presented  to  either  crewmember  through  the  Helmet- 
Mounted  Display  (HMD)  and  on  the  panel-mounted  cathode  ray 
tube  (CRT)  displays.  The  HMD,  which  is  p^rt  of  the  Inte¬ 
grated  Helmet  and  Display  Sight  System  (IHADSS),  presents 
visual  information  to  the  crewmember  on  a  one-inch  diameter 
screen  attached  to  the  helmet,  providing  a  30°  (vertical)  by 
40°  (horizontal)  field  of  view.  The  HMD  enables  the  crew¬ 
member  to  cross-check  flight  and  weapons  information  while 
viewina  the  external  visual  scene. 


The  27  different  symbols  that  compose  the  flight  symbol 
set  can  be  presented  to  assist  the  crewmember  in  flying  the 
aircraft.  A  slightly  different  subset  of  the  symbols  is 
presented  during  hover,  transition,  cruise,  and  bob-up  modes 
of  symbology.  The  17  different  symbols  that  compose  the 
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weapons  symbol  set  can  be  presented  to  assist  the  crewmember 
in  operating  the  weapons  system.  Fourteen  symbols  are  common 
to  both  the  flight  and  weapons  symbol  sets.  The  symbols  vary 
in  size,  shape,  location,  and  the  manner  in  which  they 
functionally  represent  the  status  of  the  aircraft  or  weapons 
system  elements.  Information  about  the  aircraft  or  weapons 
systems  may  be  represented  by  changes  in  symbol  size,  posi¬ 
tion,  or  rate  of  movement.  The  number  of  symbols  displayed 
at  any  given  time  varies,  depending  on  the  nature  of  the 
flight  or  weapons  task  and  the  selected  symbology  mode. 


Ne.ed 

To  become  fully  qualified  in  the  AH-64  attack  heli¬ 
copter,  a  student  aviator  must  learn  to  recognize,  under¬ 
stand,  and  interpret  the  symbology  presented  on  the  heli¬ 
copter's  visual  displays,  and  to  integrate  multiple  sources 
of  information  appearing  on  the  displays.  During  the  AH-64 
Aircraft  Qualification  Course  (AQC) ,  student  aviators  are 
taught  to  use  the  symbology  through  classroom  lectures, 
videotape  presentations,  self-study  handouts,  and  technical 
manuals  containing  static  diagrams  of  the  symbology. 
Opportunities  for  additional  practice  with  the  display 
symbology  are  available  on  three  training  devices:  (a)  the 

TADS  Selected  Task  Trainer  (TSTT) ,  (b)  the  Cockpit  Weapons 

Emergency  Procedures  Trainer  (CWEPT) ,  and  (c)  the  Combat 
Mission  Simulator  (CMS) . 

The  TSTT  is  a  part-task  trainer  designed  to  support 
initial  CPG  qualification  and  refresher  training  in  e  AH- 
64,  and  to  support  TADS  skill  sustainment  during  mission  and 
continuation  training  in  field  units.  It  provides  practice 
only  with  weapons  symbology.  The  CWEPT  is  a  full-scale  crew 
station  procedures  trainer.  It  is  designed  to  provide 
training  to  the  pilot  and  the  CPG  in  both  normal  and  emer¬ 
gency  flight  procedures  and  normal  and  emergency  mission  and 
avionics  equipment  operation.  The  CMS  is  a  six  degree-of- 
freedom  motion-based  simulator  designed  to  simulate  the 
flight  and  weapons  capabilities  of  the  AH-64  aircraft.  It 
currently  is  used  to  provide  training  in  combat  mission 
scenarios  during  the  Combat  Skills  phase  of  the  AH-64  AQC  and 
during  operational  field  unit  training. 

The  training  designs  of  the  TSTT,  CWEPT,  and  CMS  do  not 
include  training  on  basic  symbology  identification  and  inter¬ 
pretation.  They  assume  a  substantial  degree  of  familiarity 
with  flight  and  weapons  symbology.  Training  basic  symbology 
skills  in  the  available  devices  consumes  an  unacceptably 
large  amount  of  time  that  could  be  spent  more  productively  on 
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specialized  training.  Furthermore,  when  training  in  the 
TSTT ,  CWEPT,  or  CMS,  the  students  typically  do  not  have  an 
opportunity  to  use  the  AH-64  display  symbology  under  the  full 
range  of  missions,  modes,  weapons,  systems  options,  and 
system  failures. 

Therefore,  the  Training  and  Doctrine  Command  (TRAD OC) 
System  Manager  (TSM)  for  the  AH-64  requested  that  the  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
(ARIARDA)  develop  and  evaluate  a  training  module  to  train  AH- 
64  student  aviators  to  understand  and  interpret  flight  and 
weapons  symbology. 


The  Statement  of  Work  (SOW)  for  this  project  sets  forth 
six  specific  requirements  for  the  training  module.  Briefly, 
the  training  module  should: 

•  be  designed  in  a  self-instructional  format  (i.e.,  not 
require  an  instructor  pilot  (IP)  or  other  instruc¬ 
tional  personnel)  and  enable  training  to  be  conducted 
in  a  classroom  setting; 

•  cover  the  full  range  of  aircraft  missions  and  weapons 
system  options; 

•  be  capable  of  storing  performance  data  and  providing 
one  or  more  indexes  of  performance  at  the  end  of  each 
training  exercise; 

•  provide  immediate  feedback  and  remedial  instruction 
when  errors  occur, 

•  be  suitable  for  both  initial  skill  acquisition  in  an 
institutional  training  setting  and  skill  sustainment 
training  in  a  unit  training  setting;  and 

•  be  flexible  enough  to  allow  revisions  resulting  from 
(a)  changes  in  the  aircraft,  (b)  changes  in  the 
avionics  system,  or  (c)  deficiencies  in  the  training 
module  revealed  by  formal  evaluation  and  feedback  from 
the  user. 

In  addition,  the  SOW  specifies  that  the  training  module 
should  be  designed  to  augment,  but  not  replace  existing 
training  devices,  and  should  not  require  the  fabrication  and 
use  of  mockups  or  other  costly  training  aids. 

Accordingly,  the  first  project  objective  is  to  develop  a 
training  module  suitable  for  (a)  training  AH-64  A QC  student 
aviators  to  identify  and  interpret  flight  and  weapons 
symbology,  and  (b)  providing  skill  sustainment  training  to 
aviators  in  a  field  unit  setting.  The  second  objective  is  to 
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design  and  conduct  an  empirical  evaluation  of  the  training 
module  in  both  an  institutional  and  a  field  unit  setting. 


Ten  research  tasks  are  required  to  accomplish  the 
project  objectives.  The  research  tasks  are  the  following: 

•  review  the  relevant  literature, 

•  identify  the  flight  and  weapons  tasks  requiring  the 
use  of  symbology, 

•  interview  subject  matter  experts  (SMEs), 

•  identify  the  capabilities  and  limitations  of  existing 
training  devices, 

•  define  the  scope  of  training, 

•  define  the  training  approaches  and  settings, 

•  define  the  terminal  learning  objectives, 

«  identify  and  evaluate  the  training  media, 

•  design  the  prototype  training  module,  and 

•  evaluate  the  prototype  training  module. 


Work  began  on  the  project  in  December  1986.  Discussions 
with  SMEs  suggested  that  performance  deficiencies  in  sym¬ 
bology  usage  exist  in  the  following  areas: 

•  recognizing  and  interpreting  the  meaning  of  symbols 
presented  alon<~  and  in  the  context  of  other  symbols, 

•  interpreting  the  meaning  of  symbology  movement, 

•  correctly  associating  switch  actions  and  control  move¬ 
ments  with  static  or  dynamic  symbology,  and 

•  selectively  attending  to  and  interpreting  display 
symbology  when  the  symbology  is  superimposed  on  a 
dynamic  external  visual  scene. 

At  the  end  of  the  first  contract  year,  the  general 
content  and  organization  of  the  training  module  has  been 
determined.  A  preliminary  decision  was  made  to  organize  the 
symbology  training  module  into  two  parts.  Part  I  covers 
basic  symbology  usage  skills  and  addresses  the  first  two 
deficiencies  cited  above;  Part  II  covers  advanced  symbology 
usage  skills  and  addresses  the  last  two  deficiencies. 


Part  I  of  the  training  module  is  organized  into  five 
separate  lessons.  Each  lesson  is  designed  to  be  self- 
contained  and  presents  instructional  material  and  quizzes  on 
the  following  groups  of  related  symbols: 

•  position/movement  symbols, 

•  attitude/altitude  symbols, 

•  heading/navigation  symbols, 

•  cueing/reference  symbols,  and 

•  weapons  delivery  symbols. 

Work  currently  is  underway  to  develop  instruct ional 
materials,  quizzes,  performance  measures,  and  software 
branching  logic  necessary  for  the  student  to  progress 
smoothly  and  effectively  through  the  lessons  and  quizzes. 

The  softvrare  and  hardware  required  to  deliver  the  instruc¬ 
tional  material  in  an  efficient  and  effective  manner  has  been 
identified.  The  training  module  is  being  written  in  compiled 
BASIC  and  is  designed  to  run  on  a  Zenith  PC  AT-compatible 
microcomputer  equipped  with  one  megabyte  of  random  access 
memory  (RAM) ,  a  20  megabyte  internal  hard  disk,  an  Enhanced 
Graphics  Adapter  (EGA) ,  and  a  high  resolution  EGA  color 
monitor.  Additional  software  and  hardware  enhancements  will 
be  considered  as  the  goals  of  the  training  module  become  more 
clearly  defined. 

At  the  end  of  the  contract  year,  preliminary  storyboards 
for  the  first  three  lessons  in  Part  I  have  been  developed. 
Software  routines  for  presenting  symbols  on  the  computer 
screen,  individually  and  in  combination,  and  for  accurately 
depicting  symbology  movement  are  being  developed  and  tested. 
An  operational  version  of  the  first  lesson  has  been  developed 
and  is  being  reviewed  by  selected  aviation  SMEs. 


Work  Projected 

Part  I  of  the  training  module  is  scheduled  to  be 
completed  and  ready  for  evaluation  by  the  end  of  December 
1987,  and  Part  II  is  scheduled  to  be  completed  and  ready  for 
evaluation  by  the  end  of  February  1988.  The  training 
effectiveness  of  the  module  in  an  institutional  training 
environment  will  be  evaluated  at  Fort  Rucker  during  the 
second  and  third  quarters  of  calendar  year  (CY)  1988.  The 
training  effectiveness  of  the  module  in  a  field  unit  training 
environment  will  be  evaluated  at  selected  field  unit  sites 
during  the  third  and  fourth  quarters  of  CY  1988.  The  final 
version  of  the  training  module  and  the  final  project  report 
is  projected  to  be  completed  and  submitted  to  ARIARDA  during 
the  second  quarter  of  CY  1989. 


EFFECTIVENESS  ANALYSES  OF  AVIATION  PART-TASK  TRAINERS 
Dr.  Dudley  J.  Terrell,  Project  Director 


Background 

To  address  identified  training  deficiencies,  the  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
(ARIARDA)  is  conducting  research  to  develop  part-task 
trainers  that  provide  instruction  in  navigation,  weapons 
systems  operations,  preflight  inspections,  and  other  aviation 
subjects.  As  these  trainers  are  developed,  final  training 
effectiveness  analyses  (TEAs)  are  needed  to  ensure  the 
trainers  are  successful  in  meeting  the  training  objectives. 
The  final  operational  tests  are  usually  in  the  form  of  a 
transfer  of  training  study  and  serve  several  purposes. 

First,  TEA  results  will  determine  if  the  part-task  trainers 
are  effective  as  designed  or  if  changes  are  required  before 
implementation.  Second,  TEA  results  can  be  used  to  prescribe 
optimal  strategies  for  using  the  training  systems.  Finally, 
the  TEA  provides  a  methodology  for  the  continuous  evaluation 
and  maintenance  of  training  effectiveness  after  the  full- 
scale  implementation  of  the  part-task  trainer. 

One  part-task  trainer  that  is  ready  for  a  TEA  is  the 
Advanced  Map  Interpretation  and  Terrain  Analysis  Course  (Adv- 
MITAC)  developed  by  Anacapa  Sciences,  Inc.  (Miles  &  LaPointe, 
1986) .  The  Adv-MITAC  program  is  designed  to  supplement 
classroom  and  inflight  training  in  terrain  navigation  at  nap- 
of-the-earth  altitudes.  The  equipment  consists  of  a  micro¬ 
computer,  videodisc  player,  color  monitor,  seven  videodiscs, 
and  a  set  of  maps.  The  training  program  is  divided  into  13 
lessons,  each  designed  to  be  slightly  more  difficult  than  the 
previous  one. 

Each  lesson  begins  with  a  preflight  map  study  of  the 
area  of  operations.  The  student  then  views  a  film  taken  from 
the  front  window  of  a  helicopter  flying  a  route  at  low 
altitude.  At  various  points  in  the  film,  the  action  stops 
and  the  student  must  enter  the  map  coordinates  for  that 
point.  If  the  student's  coordinates  are  within  200  meters  of 
the  correct  location,  the  action  continues  until  the  next 
checkpoint.  If  the  student's  coordinates  are  more  than  200 
meters  from  the  correct  location,  a  narrated  review  of  the 
route  from  the  last  checkpoint  is  played.  After  the  review, 
the  action  continues  until  the  next  checkpoint.  Each  route 
includes  seven  checkpoints.  The  computer  calculates  the 
amount  of  error  at  each  point  and  presents  the  results  to  the 
student  at  the  end  of  the  lesson. 
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The  Adv-MITAC  package  is  ready  for  implementation  except 
for  an  evaluation  of  its  effectiveness.  This  evaluation  must 
either  provide  data  supporting  the  program's  efficacy  or 
provide  recommendations  for  improvements  in  the  program.  In 
addition,  the  evaluation  must  provide  a  basis  for  recom¬ 
mending  the  optimal  strategy  for  using  Adv-MITAC  in  the 
current  aviation  training  program. 


Project  Objectives 

The  purpose  of  this  project  is  to  evaluate  the  effec¬ 
tiveness  of  the  Adv-MITAC  part-task  trainer  prior  to  its 
implementation.  This  project  has  three  specific  objectives: 

•  develop  a  suitable  navigation  performance  measurement 
technique , 

•  provide  an  operational  test  of  the  Adv-MITAC  trainer, 
and 

•  develop  recommendations  for  the  optimal  utilization  of 
the  Adv-MITAC  program  in  the  initial  entry  rotary  wing 
(IERW)  course. 


Research  Approach 

The  Adv-MITAC  TEA  will  be  conducted  in  the  following 
four  phases: 

•  Phase  1:  Conduct  a  literature  review  of  existing 
navigation  training  programs  and  accompanying  evalua¬ 
tions,  and  develop  a  research  plan  to  evaluate  the 
effectiveness  of  the  Adv-MITAC  program. 

•  Phase  2:  Develop  a  technique  for  measuring  navigation 
performance  before  and  after  exposure  to  the  Adv-MITAC 
program . 

•  Phase  3:  Collect  TEA  data  during  the  OH-58C  Combat 
Skills  section  of  the  IERW  course. 

•  Phase  4:  Analyze  the  data  collected  during  Phase  3 
and  write  a  report  describing  the  research  method¬ 
ology,  measurement  techniques,  results,  and  recommen¬ 
dations  for  the  use  of  Adv-MITAC. 


Anacapa  researchers  began  work  on  the  project  in 
September,  1987.  At  the  end  of  the  first  contract  year  a 
preliminary  review  of  the  TEA  literature  and  methodology  had 
been  completed.  The  project  director  attended  the  Terrain 
Flight  Operations  Class  of  the  IERW  course  and  has  started 
developing  the  performance  measurement  techniques  and  the 
research  plan. 


Work  Projected 

The  technical  details  of  the  Adv-MITAC  TEA  research  plan 
are  presently  being  developed,  but  the  steps  outlined  in 
Phases  2-4  of  the  research  approach  remain  to  be  completed. 

In  addition,  other  part-task  trainers  (e.g.,  Basic-MITAC  and 
the  AH- IS  Preflight  Inspection  Trainer)  will  be  subjected  to 
TEA  research  when  the  product  development  is  completed. 
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